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ABSTRACT * ' , / 

Members of the Research and Development Exchange 
^-TfEDx) agreed to co^^laborate in. the development of a, resource base on 
the identifibation and validation of promising educational ^rogra^ms^ 
practicesr and products., state activities in validation an^d state 
concerns ab^out and reco^mendatidns concerning current nationally used 
processes are the subject ^of ^his report. Questionnaires were sent by 
'the regional ^educational laboratories to th'e states in their regipns 
concerning:* (1) the e?tent to which procedures to validate €xemf)lafy 
^programs aye being u^^d by stt^te departments of education; (2)^ the 
.nature" o^ the va^lidation process in use: (3) state implementation 
procedures for validation and the <^scope of state validation efforts 
to date: (4) s^te organizational ar rang^arents for validation and the 
nature of state ^supporlf for th^' dissemination of validated programs; 
"g[nd^(5) the^extent of stat^ c^ollabprative validation activities, kn 
overview' of current state validat'icn practices Is shown on a chart- 
SuT^ re suit Sr organized by -j^egion^ are presented in 17 tatles 
a%c^niij)anied b.y descriptions- A brief section discusses concerns and 
rsgoes about the validation of edji^cationai program? and practices.. In 
the^appendices ar^ summaries for each state^ a list 6t state 
pontactSr description of validation processes^ and the survey form 



* 
* 



Heproductions supplied by EDRS are the best that can be made 
* ' from the original document. 



**1^*********]|(*]|t]|t*^3|t***>|C***************^^4t3|{^^,|t,|t,|t,|t,|C,|(,|(,|c4,|C,|t,|t 



U.S, DEPARTMENT-OF EDUCATIDN 

, NATIONAL INSTITUTE OF EDUCATION 

EOUCATlONALvRESOURCES INFORMATION 
CENTER (ERIC)* 
^Ths document has been reproduced as 
received from the person or organuation 
, originating it 

J Minor changes have been made to improve 
reproduction quality \ 



' Pomts of view or opmions stated m this docu- 
ment do-rtot necessanly represent off icia* NIE 
posMion or policy *^ 



CO 

• • • 

- 

• ' • Survey of State Procedure^ 

^ ■ - • for the ValidatiojT of Educational Pfograms 

fNj ' - ' 

CD 

^ , I August 1981 

A Collaborative Effort of the 
Research .and development Exchange 



H 

O 

< 



ERIC 



Prepared by 



Ljnda Reed 
R&D Interpretation Service 
CEMREL,.Inc. 

\ - " Ed Patrick 

Regional Exchange 
, Research for Better Schools, Inc. 

. " David Holdzkom 
Regional Exchange* 
Appalachia Educational Laboratory 



The material 4n this publication was prepared under a contract with the 
National Institute of Education^ U.S. D^rtment of Education. Its con 
tents do not necessarily reflect the position or policy. of the U.S. 
Department of' Education and no official endorsement by the U.S. Depart- 
ment of Education should be inferred. 



1 



Research and Development Exchange (RDx) 



Regional Excfranqes (Rx) 



Appalachia Educational Laboratory (AELJ 
P.O. Bqx 1348 ^ 
Charleston, Wes.yVirginia 25325 
(.800) 624-9120 

Director: Sandra Orletsky 



SWRL Research and Development 
4665 Lampson Avenue' 
Los Alamitos, California 90720 
(213) 598-7661 

Director: Roger Scott 



CEMREL, Inc. 

•3120 5dth Street 

St. Louis, Missouri 63139 

(314) 781-2900 

Director: Carol Thomas 



McREl. , ■ • , 

4709 Bel review 
Kansas City, Missouri 64112 
• (816?) 756-2401 

.Director: Susan Everson 

Northeast Regional -Exchange (NEREX) 
Merrimack Educat4on Center 
•101 Mill Road 

Chelmsford, Massachusetts 01824 
(617) 256-3985 

Director: J. Lynn Griesemer 

Northwest Regional Educational 

Laboratory (NWREL) 
3G0 S.W. Sixth Avenue 
Portland, Oregon 97204 
(503) 248-6800 

Dird6;or: Joe Pascarelli 

Research for Better Schools, Inc. (RBS) 
444 North Third Street 
Philadelphfa, Pennsylvania 19123 
(215) 574-9300 

Director: Richard McCann 



H- Southwest Educational De 
Laboratory (SED 
211 East Seventh Street 
Austin J Texas 78701 
(512) 476-6861. 

Preston Kronkosk^ ; 




Director: 



Central Support Services • 

• « 

System Support Service 

Far West Laboratory for Educational 

Research aiyl Development 
1855 Folsom Street 
San, Francisco, California 94103 ■ 
(415) 565-3179 

Director: Stanley Chow 

RSD Interpretation Service 
CEMREL, Inc. 

3120 59th Street • . 

St. Lou.is, Missouri -63139 
(314) 781-2900 

Director: Linda Reed*.- • 

Resource and Referral Sler.vice 
National Center, fgr Research in 

Vocational Education 
1960 Kenny Road 
Columbus, Ohio 43210 
(614) 48e-3655- 

Director: jay'Smink 

. « ■ 

Dissemination^ Support Service 
Northwestv Regional Educational 

Laboratory 
300 S.W." Sixth Avenue 
Portland, Oregon" 97204. 
.(503) 248-6^00 - " ' - - . • 

Director: Joe Pascarelli . 



7 



ERIC 



iii ^; 



CONTENTS 



Acknowledgments 
External Reviewers 
Introduction 
Descriptive Study 
Overview Chart 

Tables 2-17, with Prose Descriptions 
Concerns and Recommendations 

• • 

Appendixes 

A. State Summaries 

B. List of State Contacts 

C. Description of IVD, JDRP, \nd 
Sharing Business Success Prpcesses 

D. Survey form • 



V 

vii 
3 

7 

15 
.27 
63 



69 

175 ^ 
183 

201 



ERIC , 



' iv .5' 





f 


., k 






• 




■ - . ACKNOWLEDfiMENTS 




i 


This report is th$ result of a coll'aborativje effort of the Research and 
Development Exchange (RDx), a national effort funded by the.National Institute* ^ ' 
of Education, Each of the eighr, Regional Exchanges (Rx) in the RDx participated ^ 
- in .the' effort. Without the Itesistance of the' Rx staff members, this report ♦ 
"WouTd have been an impossible undertaking* The labora-tories and RDx staff 
members which were involved are listed below. 


• 


r 

Appalachia Educatignal Laboratory (AEL) 

1 

* 

) 

CEMREL, Inc. 


( ' 

David Holdzkom 
Merrill Meehan 
Shirley Keene 
Lori, Roberts 




« 

Greg Goodman . . - 
Dane Manis 




Far West Laboratory for Educational 
Research and Development 


Carol Thomas 




Mid-Continent Regional Educational 
Laboratory (McREL). — 


Karen Benson 
Susan Everson 


• 


Northeast Regional Exchang,e\(NEREX) 


Richard Lavin 
Larry Vaughn 


Northwest Regional Educational Laboratory 
(NWREL) 


David Campbell p 
Kathleen'^^otton 1 




Research for Better Schools,. Inc.. - 


Ed Patrick - 


( , 


Southwest Educational Development 
Laboratory (SEDL) ' - ' 

• I 


Nancy Baker Jones 
PfestonXronkosky 
Jack LumbTey 
Anna Hundley 




SWRL Research and Development 


Jjudy Larson 


o 

1 • 

.f _ 

■il ■ 


( ^ 

A number of add.itional people were instrumental in the' undertaking. 
Cecilia DiBella, Director of. the Massachusetts Dissemination Project 
(representing the Northeast Consortium at the time), and. Clark Chipman, 

/Educational' Program Spepialist with the Division of Educational Dissemination, 
ROEP V, served on the RDx Validation Task Force. Each of^the state people who 
is identified as a state contact, both in the' state summary and In the list of 
state contacts (Appendix B), was essential to the prpcess. These SEA 
representatives completed the initial survey i read and often carefully revised 

' their state summary* and had endless "patience with'^quegtions and requests for 
.clarification called in by telephone. Finally, the people who served as 
externeri reviewers for the document offered valuable suggestions for improve- 
ment. The external reviewers are listed on page vii. ' / 


• ERIC 


_ • . - ■ ,v . . 


> 

■a V 

\ 



ROIS staff have in many ways •served as the backbone of the overall 
effort in validation. Karen Temmen served on the Validation Task Force 
and aiordinated the development of the resource file^ Sandy Ruder and , 
Ma^y Ann Isaacs were responsible, respectively,, for production" of all 
documents related to the effort and abstracting* and indexing of the file. 

Linda Reed 
Director 

. ^ • .R&D Interpretation Service 

\ 



C 



EXTERNAL k^VIEWERS 



Mr. Peter Bachmann 

State. Facilitator 

Delaware Department of Public 

Instruction 
P.O. Box 1402 . 
Bover, Delaware 19901 , 



Dr. George E. Brehman, Jr. 
Pennsylvania Department of Education 
Box 911 ■ 

Harrisburg, Pennsylvania 17126 



Ms. Virginia Cutter 
261-6 Spring Lane 
Austin, Texas 78701 



Mr. Gene Dickson 
Director, Title IV-C and 

State facilitator 
Department of Elementary -^nd 

Secondary Education 
Kneip Building 
Pierre,.. South Dakota 57501 



Mr. George Harris , ' - 
Ohio Departments Building 
65 S. Front Street, Room 804 
Columbus, Ohio 43215 • 



Mr. George Katagiri 
Oregon Department of Education 
700 Pringle Parkway, S.E. 
Salem, Oregon 97310 



Dr. Richard Keene 

Specialist in Research Design , 

and Statistics \ 
Evaluation and Planrrtflf-" 
Utah State Office of Education 
250 ?E. Fifth South Street 
Salt Lake Ci-ty, Utah 84111 



\ 



Mr. Michael kuhn * . 

Department of Educa^tibn 
Knott ^Building 
Tallahassee, Florida "32304 



Mr.- Ronald E. Lesher 

Director, Program Systems'" 

New Jersey Department of Education 

Division of Operations, Research, 

and Evaluation 
P.O. Box 2019 

Trenton, New Jersey 08607 



Mr. Charles J. Nelson 
Oregon Department of Education 
700 Pringle Parkway, S.E. 
Salem, Oregon 97310 



Mr. Tom Odneal 
Director, Title IV 

Missouri Department of Elementary and 

Secondary Education 
Box 480 

Jefferson City, Missouri 65T01 . 



V 



Col. Francis^T. Phillips, Jr. 
Coordinator of Special 'Programs 

and Grants 
Virginia DepartJipnt of Education • 
Box 6Q , \ • 

Title IV-C Office 
Richmond, Virginia 23216 



Mr. AT Ramirez 

. Nevada State Department of Education 
Capitol. Complex 
Carson City, NevlTda 89710 

-V - " ( 

Mr. Richard Riley • / 
Title ESEA 

Maine, Department of Educational and 

Cultural Services 
Education Building, Mail Station 23 
Augusta, Maine 04333 



\ 



EXTERNAL REV I EWERS^ (Continued) 



Mr.,>Jarod Shady 
State Facilita.tor 
Loraco Plaza' 

Concord, New Hampshire '03301 



\ 



- Ms. Ann Shenassa 
.345 Education Building 

• University of Illinois 
Urbana, rilinois 61801 



Dr^ Beverly Wheeler ' • 
Director^ of Information Dissei^iination 
Arizona State Department of Education 
1535 West Jefferson 
Phoenix, Arizona 85007 • 



Mr. Rpbert Workman 
USOE Region X • 
Arcade Plaza Building 
1321 Second\Avenue 
Seattle, -.Washington 98101 



r 



viii- 



t 



, ' Survey of State Procedures for the Validation 

■ of Educational Programs 

INTRODUCTION '> " •;' 

The Research and>Pevelopment Exchange (RDx) is a network of regio'nal, 
educationa^l'^aboratories and university-based research and devilopmeht centers 
working to support state and local school "Imprcvemertt effort^. Initiated by ^ 
the National Institute of Education (NIE) in 1976,vthe* RDx 'is, Composed. of 
eight Regional Exchanges (Rx) and foii.r- central .sjpport services. \ 

In 1979. meiibers of the RDx agreed to col labor jfte^^n the dfivelo^ent of a 
resource base on the identification and validation of promising progr^ams,. / 
practices, and products. It had ^become apparent to sev^rkl of tKe Exc)iahges 
that the identification of high.qaality educatio)^aT progr*aiii^ was a matter of 
considerable concern to many of the states in their region^, a concern wht^h 
surfaced at the same time that the U^S. Office of Education was introducing 



_^ revised ve'rsion of the Identification, Validation," Dissemination ('IVD) 
iJrocess. Three Exchanges have, at the^request of their client^, conducted 
conferences on validation, and a fourth Exchange is plannino^stjch a 
conference. The collaborative effort, it wds" decided, would include 
development of a central file of resource materiaU wi^tff^m^'^companying 



annotated bibliography,, development of a synthesis of ^h^^^Atei^^t^^^ 
validatiort, ar^ survey of State activities in validation of edi^tional 
programs, practices, and products, and development of a If^t of state-ba^ed 
concerns and recommendations about validation. 
Our goals*- for the collaborative effort were 

T. to create a centralized information base^o which the Regional . 
• Exchanges aiid the-ir -clients would have access; to be housed at CEMREL 



' and managed l?y the Research and Development Interpretation Service 

(RDIS). . . ' . . 

'2. To assj^t, where -appropriate; state planning,' refinement, and/or 

implementation of validation {)rocedurQ5 by creating a <^3ool of 
• information about -states* activities* -.including management systems, 

training procedures, criteria used by validation' teams,' and 

- procedures for supporting and" dissemirratlng "information about ^ 

' ' - ' . ' - • ^--1 ' - 

validated programs and practices. - / ; ' 

^ : < , ' • •. 

3. To attempt to influence- the future agenda of the , Department of 
Education wjth regard to ,the validation of educational program's and 
practices* . ' 



This volume constitutes the-reWt on state activities in -validation and 
on 'their concerns about and recommendations* with respect to* current nationally 
used processes. The survey study, was designed to^dentlfy:" (1)" the extent to- 
which procedures to validate exejnplaj^y programs ar^ being used by .^tate 
departments of education inr the fifty states; (^). the nature of the validation 
process in use; ('3) state imp lementatloa procedures for validation'and -the 
scopfe of state validation efforts^^date; (4) vstate organizational 



arrangertients* for, validation ar^d the nature of state support for the. 
dissemination of- validated programs; and/(5) .the extent of state collaborative 



Vial idati«n activities. - • , ^ ^Be*. N 

The study was conducted^in 1980-1981 by the Regional Exchanges- arid RDIS. 
l^DIS staff were primarily'respo'nsible for the design and overall management ' of . 
the survey. Each of the -Regional Exchawes assisted with the collection of ' 

. . - . • • • ♦ ,x • ■ . . 



data from state staff in'their respective regions and also served as a member 
of the RDx Task "Force on Validation. ^ ^ 

To the best of our knowledge, this survey -is th'e second study which has 
obtained (detailed information about state activities in validation. A similar 
survey was conductedin August 1977 by Ray E. Foster of the Educational 
Innovations Section of the Kldfida Department of Education.* The RDx survey, 
designed to offer information about state activities in more detail than the 
Florida report, shows that the' state of ^fairs has not^ changed a great deaV 
since 1977. ' _ ^ 



*Ray E. Foster, Sur'vey of Certification Methods Applied to Innovative Public 
School Programs (Tallahiassee, Florida: Educational Innovations Section, 
•Florida Department of Education, 1978). ' 



^ DESCRIPTIVE STUDY . 
This section of the report will present thjB purpose and method relevant to 
the "Questionnaire to IdentifyyState Procedures for the Validation of 
Educational Programs, Practices, and Products" employed in the: descriptive* 
study of^state validation activities. ' ^ 



Purpose " " . . 

The descriptive study was designed to identify (1) the extent to which 
procedure? to validate exemplary programs are being used by state department? 
of education in the fifty, states, (2) the^ature of the procedures used for 
validation (whether nationally or state developed), (3) how state staff,' 
typically' organize^theinselves to implement validation policies and procedures, 
.(4). the extent and nature of state support of pre- and post -validation 
activities," and (5) the extent to which states collaborate with one another. 



The Questionaire 

The "Questionnaire to Identify State Procedures for the Validation of 
Educational Programs, Practices, and Products" (see Appendix D) was-divided 
into several sections. The first asked for specific information about the 
person cc/mpleting thQ questionnaire. This information was considered 
important because the group conducting the survey felt that these people would 
be called upon for additional information as the results of the survey were 
compiled. Such was 'indeed the case— con'tact people were. calle^ upon several 
times to supply additional information and to review material -specific to 
their state fi^ee Appendix B for a complete list of state contacts). . 



The first question on the questionnaire asked respondents whether the' 
State has a formal method for. validating, or certifying the effectiveness^ of, 
educational programs and practices. Respondents who answered "no" wei^e asked 
to complete answers 2-ld* Questions 2-4 were designed to\ determine whether 
the sjfcate^had ever had a formal procedure, when it had been in effect, and why 
it hid been discontinued. Question 5 asked whether the-state'has a promising 
praj(itices file'. Respondents^ answering "yes" were asked to complete questions 
6-10, which el^ited information about the state|s procedure 'for identi^ijg, 
promising practices and managing .the promising practices file. 

• Questions 11-22 on tiie questionnaire*. asked respondents to descri-be in 
detail the validation procedure used by the state, the crU^najjpon which 
applications for validation wfere judged, the composition of validation, teams, 
.training procfedures for validators, responsibilities of the state and local . 
education. agency for support and dissemination of validation programs and 
projects, state mechanisms for assisting programs which are nat approved for 
validation, and state policies concerning re-valid^tion. . 

The purpose of the questions was to elic4t detailed information about 
state policies and procedures so that an across-state comparison could be 
.made. Respondents were encouraged to attach state-develdped procedure 
jnanuals, application forms, criteria, and so on, so that the information could 
be as complete as possible and so that they might ^save time in completion of 



the questionnaire. 
Questions 23-2 

staff in the state departmer\t of education who werfe responsible for ^ 



Questions 23-26 asked for information about the number of professional 



/ 



Validation activities, the number of applications for validation submitted on 
.a yearly basis, the number of programs and projects validated, on the average, 
per year, and the 'Schedule for completion of ' the processing cycle. The 
purpose of these- questions was. to identify theSiVerall level of effort, by. • 
region and national Ty^ that states are devoting to validation activities and 
the number of, validated programs that result from the effprt. 

Question 27 Vsked respondents to identify the ways in which the state ^ 
collaborates 'with other states in the area of validation of educational 
programs and practices. The question was designed to identify. the extent to 
which s-tates support interstate adoption of 'validated programs, use in1;erstate 
teams of validators, or use a snared validation -procedure.' . 

Questions 28 and 29, the last two questions on the survey form, arsked 
respondents ta indicate thjfeir levelof satisfaction with the procedure used by 
their state and to identify concerns and issues about .validation which they 
would' like to see brought to.^e attention ol^^federal government officials 



involved in the development artd implementation of federally sponsored 

r 

>. The purpose of these. questions was to determine the 
extent to which ^he collective attitude of the Respondents was favorable or- 
unfavorable with respect to' thirst atus of valictationCf'educational programs 
and practi'^es in the Onited States. In addition, the group conducting the 
survey was interested in identil^ying common concerns and issues which should 
be brought to the attention of federal officials. 

Method ' . 

This section of the report will describe the sample, procedure, and data' 
analysis process Involved in the survey study. 



. The collaborative team and their responsibilities. In May 1979, several 
Regional Exchanges (Rx) within the Research and Development Exchange (RDx) 
agreed to collaborate in the development of a resource base coHs^stihg of both 
theoretical documents on validation and up-to^ate descriptions of current 
^practices- in the identification and validation of exemplary programs and 
practices. A major part of this effqrt, they agreed, would be an analysis of 
state activities in the area of validation of exemplary programs and 
practices* The Research and Development Interpret at idn*. Service (RDIS) agreed 
to coordinate the collaborative effort, and each cboperating Rx identified a 
staff member who would represent the Exchange and its region in the effort* 

RDIS was responsible for the development of the "Questionnaire to Identify 
State Procedures for the .Validation of Educational Programs, Practices, ar^d 

I V 

ProdQcts." ,Rx staff were responsib,le for identifying appropriate state 
education ^agency personnel in their region whovcouid be asked to complete the 
questionnaire, for coriplling the results of the survey for their region, for > 
making follow-up contacts with the respondents as necessary to acquire 

ditional information^r clarification and to have them review materials * 
relevant to their. state, and for reviewing the various drafts of the state 
analysis* ' j ^ 

- Sample* Cooperating Exchange staff were responsible for .identifying t\\^ 
sample for the descriptive siudy. Each Rx- staff member Identified the person 
in each state in the region who was'^most knowledgeable about validation 
policies and^ procedures in tha% state.. The majority of respondents were in 
state departmfent units or divisions- responsible for and in large part 
supported by ESEA Title IV-C money. The titled of those divisions ranged from 



J 



Title IV-C*to Office pf Planning and Evaluation to Office of Program 
Improvement to Division, for Development* Most of the respondents are involved 
in vaTidfition activities for only a small portion of their time. 

Procedure . In February 1980, each Exchange received a packet of materials 
from RDIS which ir/itiated the collaborative effort on validation. The packet 
included a copy bf the. survey for each of th^ states served by the Rx. Rx 
staff were to decide how 4i^st to acquire the information necessary to complete 
the questionnaires. Following is a summary of how the information was • 



gathered. 



ERLC 



Appalachia. Educational Laboratory 
' (AEL) 



CEMREb, Inc. 



Mid-Continent Regional Educational 
Laboratory \(McREL) V 

Northeast Regional Exchange 
(NEREX) 



Northwest Regional Educational 
Laboratory (NWREL) 



Resegir-oh-^or Bettter Schools, Inc. 



Mailed out the surveys; followed 
with a letter and personal 
contact by. state consultants; 
confirmed the information by 
sending out completed 
questionnaires and following up 
with a phone call. ^ 

Analysis of state procedures 
from file information was 
followed by a phone c^ll > 
interview; confirmed the 
information by sending the 
complfetjed questionnaires and 
following up with a phone call. 

Phone call interview; confirmed 
the -information via a* phone call. 

Phone call interview; confirmed 
the information by personal 
visit and by phone. . 

Phone call to state contacts; 
mailed out surveys; clarification 
by phone calls. 

Mailed out the surveys and ^ 
discussed the task with state 
staff "^via the phone; followed 
up with an in-Kiepth telephone 
interview upon receiving the 
initial response to obtain 
elaboration on the responses. 



n' 17 



Southwest Educational Development 
Laboratory (SEDL) 



SWRL Research and Development 



Mailed out the- surveys to 
designated per sorts in"^ the state 
departments of education* 
FollowHjp telephone calls were 
made where. necessary ^n order 
to elaborate on or veril7 
information returned. 

Mailed put surveys to state 
contact's. 



The survey questionnaires were completed between March 1980 and February 
1981. All information on the questionnaires was confirmed during the months 
of January and February 1981. Table 1 shows the nunter of states, by region, 
•which were surveyed and the responses received. 



Tabit 1. N«btr of sUtet surv#ytd by r*9lon, and tht nuriwr of sUtes with validation procedures 



teqion 


[lo* of states 
In Reqion \ 


No. of SUtts 
Surveyed 


. Ho. Of SUtes 
Respondinq 


No* d/ states 
Responding with 
Validation Procedure 


No* of States 
Plaiming A 
Procedure 


No. Of SUtes 
With No, 
Procedure 


Aa 


' 9 


9 


9 


8 




k 


CEMKl 


8 * 


8 


8 


8 " 


> 








6 


6 




* 




NCKCX 


7 


7 ^ 


7 • 


7 






NURa 


6 


6 




5 


' 1 • , 




R8S 


4 


4 


4 • 


4 






sen 


6^ 


6 


6 


4 


2 






4 


4 


4 


3 
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Data analysis process . After initial completion of the survey 
questionnaire, Rx "staff compiled the information from the states in their 
region. In October 1980, the RDx Task Force on Validation of Educational 
Programs and Practices met in Boulder, Cplorado, to compile the data from 
thos^Rxs which had. completed the survey of their istates, to identify areas 
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18 



where the data fieeded further clarification, and to identify additional steps 
in the process. A subgroup of the Task Force was identified and given the_^ 
responsibility of writing the first draft of the state analysis. 

In 6arly March 1981, the Task Force subgroup .met in Charleston, West 
Virginia, to continue the compilation of the data from the state surveys and 
to draft the following sections o;^ the state analysis: Introduct^pn, Survey 
Study, Results, and State Summaries. The group identified areas where 
addition:al data clarification was needed and returned.the surveys and first 
draft of the analysis to Rx staff for further clarification and review. Rx 
staff sent each state contact the summar)^for their state for review, and if 
necessary, completion or clarification (see Appendix K for tlje state 
summaries). State contacts also re'ce^Ved 'a copy of the first draft of the 
analysis and were asked to make suggestions for revision. Rx staff were also 
asked to fill in,' and confirm with their states, ^n overview chart of state 
activities (pages 15-23). _ * 

In July 198U the complete first draft was sent -to a panel of external 
reviewers selected by the Region|^ Exchanges. The responses of the'^revi ewers 
were studied by the members o^\the Validation Task Force and; where 
appropriate, incorf)orated in the final draft. f ' 




13 

— w- 



. Chart 1 



'. OVERV-IEW OF CURRENT STATYALIOATION PRACTICES 

The survey results are summarized in Chart 1, an Overview of Current 
State Valfdatton Practices, on the pages that follow. The chan is broken 
into two parts: Part 1 ident|jfies the validation process used by each 
state and state implementation procedures and validation efforts to date; 
Part' 2 identifies state organizational arrangements and support for ■ 
dissemination of validated programs. To facilitate a 'Better understanding 
of-the- charts, brief descriptions -of the Identificatibn, Validation, 
Dissemination (IVD) process, the Joint Dissemination Review Panel (JDRP) 
process, and the Sharing Business Success proces*can be^found in Appendix V. 
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Chart 1. OVERVIEW OF CURREHT STATE VALIDATION PRACTICES. > 

Part 1 ^ 



C « Combination 
V - Variable 
0 « Optional 
SF - State Facilitator. 
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Chart 1. OVERVIEW 01^ CURRENT STATE VALIDATION PRACTICES 

Part 1 (continu^) 



C * Comb^ntftlon 

V - Variable 
, 0 " Optional 
SF « State Facilitator 
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VALIMTIOK PROCESS USED 


^ 


STATE IMPLENENTATION PROCEDURE AND VALIDATION EFFORTS TO DATE 




4 

ssMi ma%$ 

IMIfM suit •MfM 

Nf itt* Jitf 


lit lUU 


H^lkaiKs fitll tf suit rantt' 




IP 


WliMllfM KWMU 
SH tettt 


siAff vmm m 
mmm.m\n Homm 

Mrnmtf 
^ IkMifmt Syilantk 


• 

m-vimum 


RI 


• 


• 




• 




• 


• 


• 


SC 


• 


• 


• c • 






• 


• » 

• 




SD 


• 


• 




• 




• 


• 




TN 


• 






• 




• 


• 


« 


TX 


• 


• 


• • 










• 


UT 




• 


• • • 


• 


• 


• 


ft 


• 


VA 








• 


• 


• 


1 

m 


• 


VT 


• 


• 


• • • 


• 


• 






• 


m 


• 


• 




• 


• 




• 


• 


WI 


• 


• 


• f 




• 


• 


• 




wv 


• 


• 




• 


• 


• 


• 


• 


HY 

c 




7 

« f • 


r— 

• 


• 


• • 


• 


• 



ERLC 27 



28 



r4 



/ 



Chart 1. OVERVIEW OF CURRENT STATE VALIDATION PRACTICES 

Part 2 



C > Ccnblnatton 
V - Variable 
0 > Optional 
SF « State Facilitator 
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STATE ORGANIZATIONAL ARRANGENENTS AND SUPPORT FOR OISSEHIHATION OF VALIDATED PROGRAMS 
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Chart !• OVERVIEW OF CURRENT STATE VALIDATION PRACTICES 
.Part 2 (continued) 



C « Combination 
V « Variable 
0 « Optional 
SF « SUte Facilitator 
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. STATE ORGMtlZATIONAL ARRANGEMENTS AND SUPPORT FOR DISSEMINATION OF VALIDATED PROGRAMS 
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RESULTS OF SURVEY: TABLES AND DESCRIPTIONS 



The following. pages represent the results of the survey, organized by 

they serve are as follows: 



/ 



region. The Regional Exchanges and the states 
, , Appalachia Educational laboratoryJj^EL) 

' A. CEMRELi I^^c. " i , • 



Mid-Continent Regional Educational 
Laboratory (,McREL) . -• 

k * " 

\ * # ' 

\ 

•northeast- Regional Exchange (NEREX) 

- Northwest Regional Educational 
Laboratory (tWREL) 

. Research for 'Better Schools, Inc. 



Southwest Educational Development 
Laboratory (SEDL) 



SWRL Research and Development 



Alabama, Florida, Georgia, 
Kentucky, North Carolina, -° 
South Carolina, Tennessee, . 
Virginia,^ West -Virginia 

ininols, Indiana, Iowa, 
Michigan, Minnesota, 
Missouri, Ohio, Wisconsin 

Colorado, 'Icansas, Nebraska, 
North Dakota, "South Dakota, 
Wyoming 

"Connecticut, Maine, 
Massachusetts, New Hampshire, 
Jlew^rk, Rhode Island, Vermont 

■^aska-, Hawaii r idaho, 
jntana, Oregon, Washington 

Del aware, j4ry land. New (Jersey, 
Pennsylvania »■ 

Arkansas ,' iioui ana , . 
Mississippi, Jlew Mexico, • 
Oklahoma. Texas - ^ 

« 

Arizona, California, 
Nevada, Utah , 
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Table 2. Number of States Using Various Validation Practices 
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— ^ ■ ^ i 
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State Variation 
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y ■ . 

state Variation ' 
of JDRP 


State 
Developed 
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\ \\}\) , 




2 (AZ» CA) 
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(N»50) • 
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*Shar1og Business Success 
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ire than one 



* •\,**fo'*' "ch response. It vas possible for a^^tate to respond In ,a way that would place the state in ie 
category. In a^ cases, the responses of the states were Interpreted literally. . /• . 
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TABLE 2 



Nunter of States Using Various ^Validation Practices 



In ^unwary, the data in Table 2 iiyjicate that the great majority of 
states are presently irjyolvfed in validatina educational programs^'and 

. practice's* \Approxiniately 46 percent (N=»23) of the states use the USOE " 
recommended Identification, Validation, Dissemin^ation (IVD) procedure* . 
Another 12^ percent of the states (N«6) reportedly ^jse a stafeJ^ variation of the 
IVD prooesB* Thus, over half of the states surveyed are Involved in the IVD 

-process or some variation thereof^ 

Interestingly enough, 30 percent (N«15) of the states use state developed 
processes for validating educational programs and practices* Overall^ less 
than 10 percent of th^ states (N»4) use exclusively variations of the Joint 
Dissemination Review Panel (ODRP) 'process. .Finally, 10 percent of the«states . 
(N«5\ are either not involved in validating educational- programs at this time 
(Alabama and Nevada) or are in the planning stages for iinplementing ja 
valtdation progr^am (Hawai V-toii^siana, and Mississippi).. Alabamauldes 
maintain a resource file eft promising programs and practices, and Nevada has a 
procedure for identifying*^ educational projects and programs as exemplary. 
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Table 3, Number of States Using IVD/pr Other Criteria 



Region 



AEL (8) 
CEMREL (8) 
MdREL (6) 
NEREX (7) 
NWRE)L (5) 
RBS (4) 
SEDL (4) 
-SWRM3) 



^IVD 
Criteria 



Variation of IVD Criteria 
or State-Developed 



5 (FL,GA,Nq,SC,VA) 

6 (IL,IN,IA,MN,MO,WI) 



3 (KY,TN,WV)'' 

2 (MI, OH) 



6 (CO,KS,NB,ND^SD,WY) 

3 (MA,ME,RI) 

2 (IDjMT) 

2 (DE,NJ) / 

2,(AR,0K) 

A-^m 



National Totals 27 
Pef cent age 

of TQtariI^45) 60 



4 (CT,NH,NY,VT) 
3 (AK,OR,WA) 
2 (MD,PA) . 
2 (NM,TX) 
-^UT-,CAtAZ^ 



19 



42 



1 •■ 
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TABLE 3 

Number of States Using IVD or Other Criteria 



The IVD process validates programs by two primary criteria/ Programs are 
vjilidated which are educationally effective and which are exportable . Evidence 
of effectiveness is not prescribed butrtypically may include gains on' 
standardized achievement tests or other quantitative data collected on 
reliable and valid instruments. 

.Exportabllity is somewhat more subjective. Evidence from replication 
sites- is used if available. The expertise and experience of the site review 
team Is often used to meet this criterion, out programs submit infprmat ion 
concerning population serve<U~resources required, and special conditions which 
affect the implementation of the program. 

In addition tp these criteria, the program seeking validation must 
provide resource specifications: how much and what kinds of resources must be 
conmitted to a project for start-up, training of staff, materials, facilities, 
contracted services, and travel. Table 3 confirms that the IVD criteria are 
used by over half (N-27 or 60%) of the 45 states reporting that they have 
validation processes. 

The JORP rest^ primarily oh one criterion: evidence of educational 
effectiveness. Altjhough some 6ons1deration is. given to the transportability 
of an. innovation, the primary consideration Is an objective analysis of the 
evidence of? educational effectiveness. 

Nineteen of the states (423{) report that state-developed criteria must be 
met- in order' for the program to be validated. While these criteria, frequently 
Include the IVD criteria, other evidence might include ability to maintairt a 
demonstration site (Texas); existence of training materials (Tennessee); or 
evidence that the program is cost-effective (Kentucky). See the state 
summaries for other examples of state-defv^loped criteria. 
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Table 4. Number of States Using Various Training Procedures 



Region 


OSOE. 
Colorado Springs . 
(IVD) V 


Regional Office 
Training 
(IVD) 


USOE Training 
Repl icated 
In State 
(IVD) 


State Designed and 
Conducted Training 

— 


Teams Trained 
Exclusively ^ 
by Other States / 


AEL (8) 




2 (FL*,NC) 


1 (6A) 


5(KY**^NC,SC,TN,Wy) 


1 (VA) . ^ 


CEMIEL (8) 




2 (lA.MO) 


4- (IL.IN.MN.WI) 


2 (MfiOH) 




McREL (6) 


1 (CO) 


1 (SD) 


4>(C0,KS,NB,UY) 


1 (ND) 




NEREX (7) 

0 

w NWREL (5) 


2 (HE.RI) 


* 


2 (ID.hfT) 


5 (CT.MA.NY.NH.VT) , 
3 (AK,pR,.WA) s.. 




RBS (4) 




2 (DE.MD)»- 


1 (NJ) 


2 (MD.PA) 




SEDL (4) 


0 




2 (AR.OK) 

i 


"2 (NM.TX) 


- 


SWRL (3) 


1 (OT). 




t 


2 (AZ.CA) 




National 
. * Totals 


4 


V 

• 7 

; \ 

16 

* > 

• 


1 

14 


» 


1 ' 


- Percentage- 
of Total 
(N«45) 


9 


31 


- ' / 


2 . ' 



* Second-level training, replicating ROEP training, is provided in-state. . 

**Trainers trained by the U.S.. i)ff1ce of Education have created in-state training for state-developed validation 
procedure. , • . " » 
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TABLE 4 




Number of States Using Various Training ProcedQres 



Twenty-five states (56%) of the states which use a varlldatlon procedure 
train their validators in the Identification, Validation, Dissemination (IVD) 
Process developed as a national validation process by the (then) U.S. Office 
.of Education. Three (3) states which recently started the process took 
advantage of direct USOE training, which occurred most recently In 1979 In 
Colorado Springs, Colorado. That two-day conference was held to Introduce and 
offer training In the use of the newest edition of Sharing Educational 
Success: A Handbook for Validation of Educational Practices , the procedural 
tid r iubook Tor the IViVp t ^cess. — 

The USOE training Consisted of an orientation by^ Office of Education 
staff; presentations on differences between the new and' old IVD procedures; a 
case study exercise in which teams reviewed an application for validation and 
prepared an opinion regarding validation of the project; discussion of the 
project with actual sta^f from the project; reports by the teams; and meetings 
by regions to discuss ppssibilities for collaboration. 

^ ■ /■ 

In 1980-85, seven/states (I63t) had validators trained via replications of 

the national (USOE. 1979) traijiing seminars conducted by Regional Offices of ^ 

Educational Programs /(I II, IV) of the U.S.. Department of Education. The one 

way in which these training sessions differ from national trainiijig is in the 

use of panel slmul^ations Instead of actual" validated-project staff. In , 

addition, fourteen states have replicated national training seminars within 

their own stat^. 

Twenty-two states (49%) design and conduct their own training se^ions 
for validators. These sessions range from a one-hour orientatiorT session for 
a review panel (West Virginia) to a one-day workshop conducted by' the State « 
Department of Education {Arizona), to an on-site briefing for validation teams 
pridr to ever^ site visit (New Mexico). Most training sessions are. one-half 
day in length. 

' One state» Virginia^ uses te^s trained exclusively by other stages* 
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^ Table S. Number of States Uslna 1. 2. 3. 4 or 5 Member Validation Teams* 


— - *^ • 
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* . * ^- 








1 








Number on Team 








* 


Region 


1 ' 2 




4 


5 


Other 




AEL (8) 




4 (FL.NC. 
SC.VA) 


2 (6A.TN) 

• 


1 (HV) 


1 (KY-panel) . 


• 


CENRE|/ (8) 


1 (HI) 2 (IL.HN) 


.5 (IA,IN,HO, 
OH.MI) 






X- 


« 


McREL (6) > 




5 (KS.NB.NO. 

so.Wy) 




1 (CO) 


\ . ' 




NEREX (7) 


_ 2 (CT.NH**) 


3 (MA,NY,VT) 


2 (ME,RI) 




2 (CT.MI) 


Ca) 


NWREL (S) 




• 2 (ID.MA) 


1 (HT) 


2 (AK.OR) 






RBS (4) 




2 (DE.NO) 

V 


1 (MD) 




1 <PA- 
panelt 33 
meRbers) 




SEOL (4) 




4 (AR.NM,OK,TX) 








/ 


SWRL (3). 


* 


•1 (UT) 


1 (AZ) 

^ 




1 (CA- 
panelt 7 
metiers 

• 




National 
Totals 


I - 4 


26 


7 


4 


•5 .'^ ^ - 


• 


^ Percentage 
of Total 
(N.45) 


> 


•x58 

\ • • 


16 


9 • 


11 




*When we encountered a range (3-4, 6-9, etc^) we used the highw* nuinber. 








**S1te visits are optional. 
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TABLE 5 



Number of States j/slng K 2. 3, 4. or 5 
Member Validation Teams 



.Wheathe IVD process was first 1n1t1ate<Hn 1972, USOE encouraged state 
staff to use three-person validation teams composed of staff withilvaltfatlon, 
administrative, and content area/curriculum expertise. ,Tl?ese staff dvlded 
responsibilities for assessing project effectiveness, rftsource specifications, 
and project exportabHity, respectively. Data collected in the present survey 
(Table 5) Indicate that the majority of the states (26 or 58X) stm use . 
three-person validation teams and attempt to staff these teams with personnel 
that hav e ex per tis e in th e la re as indicated a b ov e , riv e states 4^yiX)- €ondtict^ — 
site visits with either one or two panel members and another elevev states 
(25%) use teams composed of> four or five members. In the former^case 
university staff witfr- evaluation and/or administrative expertise 'are usually 
employed. In the latter case teams are bolstered with state staff , state IV-C 
Advisory Committee staff, or LEA regionally based validators 1o(^ated near the 
project being validated. 

• 

Five states (nX) use validatidh panels to render final judgment on 
project validation. For a more complete explanation Qf the state panel 
' procedures* see the state summaries (Appendix A). ) . 



Table 6. Numb^ of States Using In-State or Out-of -State ^Validation Teams and Chairs . 



All ... Alf Chair • 

Region In-State Out-of -State < Out-of -State - Combination 



4 



AEL (8) . ' MKYJN,WV) 3. (QA»^ KY*,.VA**). 1 (SC) X (FUNC) 

CEMREL(8) 4 (IN,MI,bH,WI) ' 2 (IL>0) 2 (IA,MN) 

Mc REL (6) — : — = :^_(€0ySfi7WY^ : . ' 1 ( KS ) 

NEREX (7) 5 (CT,MA,ME,NH,RI) ' _ • 2 (NV.VT) 

5^NWREL (5) 4 (AK,ID,DR,WA) ^ f{m) 

RBS (4) . 2 (MD,PA) 1 M . 1 (DE) 

SEDL (4) ^ 3 (NM,OK,TX) 1 (AR) 

^RL (3) 2 (AZ, CA) 1 (UT) 



National 
/otals 


/ 

23 


?■ ^ 


♦ 


4 ' « 

6 " 


r ' 

\ ^ 


• 

Percentage . 
of Total 
(H»45) 


t 

51 : 


20 

t 




13 

ft 


18 



♦Sharing Business Success. 



♦♦Virginia will use a co(Bb1nat1on as of 1981. 
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TABLE 6 . ■ ' . y 

Number of- States : Using In-State or Out-of -State 
Valtdatiog Teams and Chairs 

All of the states which , reported having a validation procedure (45 or 
9W),use validation teams ts review the applications for validation, conduct 
on-site visits, write reports which contain their recommendations regarding 
validation, and In som? states, make the final decision. ♦ ^ 

Before, the 1979 re\^ision of Sharing Educational Success, the IVD proces? 
required the use of out-of-state validators. The r6vi^ lon7"which was designed 



to be. more flexible, encourayges the use of out-of-state validator^ but does- " 
Tjermit the use of validators from within the state.' This increased 
flexibility.h^s led a number of states which had used out-of-state valldstofs 
to discontinue^ that practice. They cite as primary reasons for the use of 
in-state validators the increasing costs involved in interstate travel and the 
greater accessibility of the people involved. \ 

A totkl of 23 states of the 45 states which have validation procedures 
{51%) report using in-state validation teams. This includes 13 of the 15 
st,ates which have developed their own validation process. Nine states (2055) 
-use out<<>of-state vaH(Jat-4on-teamsT^1x states -f^-)^^ out-of.-statr 
validation team ^chairperson, .and seven states {18X) use a combination of 
in-state and out4of -state people. 



Table 7. On-Slte Visits by Validation Teams 



L 



Region 



Yes 



.5 



' Number of Days** 
1" ,2 2.5 • 



Number of People* 
2 3 



AEL (8) 


8 


WV 


KY 


SC,TN,VA' 


PL" 


GA NC 




« 






WV 














1 — ^ 






SC,VA 






.CfMREt (8) 


' 8 


• 


MI 


lA^IL.IN.MN. 




— i: 


MI , 


11., MN . 


-IA,IN^Ma^ 














MO OH UT 


/ 








rtU LIT 








< 

C 

0 






i/c uD en 
Kb,No,SD 


/ 

* / 

7 


ND,WY 






NB,ND,SD, 
WY 


KS 




09 NtREX (7) 


7 ' 




CT,MA,NH 


ME,NY,VT** 








, NH,VT 


CT,MA,)JY 


ME,RI 




NWREL (5) 


•5. 




WA 


AK,ID,pR** 










ID.WA 


MT 


AK,OR 


RBS' (4) 


3 . 


MD 


DE 


NJ 


1 

I , 






NJ 


DE 

-WC* 


MD 




' SEDL (4) 


4 




NM,OK,TX 




/ 


AR 






AR,NM,OK, 














/ 
/ 










TX 




r 


SWRL (3) 


3 




AZ» " 


CAiUT / 






CA 


« 


UT 


AZ 




National 








/ 

/ 
















Totals 


- 43 


2 


11 


22 


3 


5 


2 


5 


25 


8 

\ — 


« 

3' 



♦When we jhcountered a range (e.g., 1. 1/?-2, 3-5), we used the hlgher^'nuniber. 
**Vennont and Oregon have 2 one-day visits. 
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TABLE r 

On-site Visits by Validation Teams 

A significant difference' between the JDR%ahd the IVD vaHdatibn 
processes is that JDRP does not permit visits %o the site of the program being 
considered for validation' while IVD mandates such a visit. 

Of 45 states reporting that validation procedures are used, 43 (96X) 
require t^a^ site visits be conducted by a validation team. .Colorado and ' 
Pennsylvania do not niake use of site vi'Sits. . " 



Of those states using on-site visits (M3), most (25 or 58X) usjavteams 
of three people, although Michigan and California use, singleV person teams-, " 
while West Virginia, Alaska,, and- Oregon send teams of five on siteyisits. 
Five (12X) states use two-person -teams and the- other eight states flSX) have 
four-pecjon teams. , . 

Site visits, for the most part,' require two days. Twenty-two states 
(51X) of the 43 using site "visits make twoniay visits. Two states (5«) make ' 
half-day site visits, and five states {12X) devote three days tojthis 
activity. Validation teams from eleven states (26X) m*ake "one-day visits to 
the-?1te and threirstater "teMr"(7X)^pW"f'ltia"T/2"aa^^H^e: ^ 



Table 8. Number of Staff at SEA Responsible for Validation 



Region 


No Staff 
Assigned 




2 


3 


4 


5 


6 


Other 


AEL (8) 




FL.NC.SC 




' "VA,WV 


KYJN 


GA 




* 


CEMREL (8) 




IA*,WI 


OH 


MN*,MO 






IN 


IL*,MI* 


McREL (6) 




CO,KSiND,SD,WY 


■ 






~\ 


NB 


• 


NERCX (7) 




NH 


ME.RI 


~ CT,VT' 


4 




NY 


MA*. 


NWREL (5) 




AK.WA 


OR' 


MT 




ID . 




' % 


RBS (4) 




DE* 


MD,NJ,PA 


■ 






• 




SEDL (4) ' 


- OK 


AR 


NM 






TX* 






SWRL (3) 


UT.AZ* • 














CA*^ 


National 
Totals 


' " -, — 

3 

. ^ t 


15 * • 


8 


7 ^ 


H 

2 ' 


• 3 


3 


4 


Percentage 
of Total 
(N«45) 


7 


33 


18 


16 






. 7 


9« 



*AZ 
CA 

' DE 



ERIC 



still developing, no staff yet 
1 FTE, 8 part-time (5-10%) 
15-20iX of one person 
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lA - one part-time'person 
IL - 1 FTE, 20-25 part-time 
MA - 6 program, 3 evaluation 



MI - 1 FTE, 20 part-time • 

MN - 3 FTE, 2 part-time. 

TX - 5 people, 2 are actively involved 



\ 
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TABLE 8 



/ NuiTfeer of Staff at S£A Responsible for Validation I 



In IS state departments (33% .of the states with a validation process),, 
one person has primary responsibility for validation activities.. Eight states 
jl8X) have two people who are responsible for'^>a44dation,^and seven states 
(16X) have three* Several states— California, Illinois, Michrfgan—assign, 
responsibility for validation to a full-time staff menter and use large 
nuntf)ers of other staff on a part-time or percent of FTE basis. 

Two states (4%) ideijtify four people as responsible for validation, three 
states (7%) use five people, and in three states (7%) six people are 
^ assigned. Massachusetts ha^s nine staff members assigned to validation. _ 




Table 9". Nurnber of Stefe that on the Average Consider 1-4, 5-10, 
11-25 or Mor^ T>^an 25 Projects for Validation Each Year 



More, Than 



Region 


1-4, 


5-10 


, 11-25 

r 


25 


AEL (8) 


2 (NC.SC) 


6* (FL.GA.KY. 
JN,VA,WV)/ 




• 


CEMREL (8) 


3 (MO^INj'IA) 


2 (MN,WI) 


• t 

3 (IL,Mj>,OI43 


• 


McREL (6) 


3 (KS,SD,WYr 


3 (CO.NB^WD) 






NEREX (7) 


2 (RI,MEV-~~ 




3 (CT,MA;NY),. 




NWREL (5) 


' 3(iiMt,or); 


2 (AK,WA) 


/ 




RBS (4) 


1 (MD) 


1 (DE) 


2 (NJ,PA) ' 




SEDL (4) 


' ^ 1 (AR) 


2 (NM,OK) 




' 1 '(TX) 


SWRL (3^* 




1 (UT) ■ 




1 (CA) 


National 
Totals 


15 


19 ' 


e 


2' 


Percentage 
of Total 
(N»45) ' 


33 

• 


42 ' 


18 


4 


*AZ - it's 


too soon to tell. 


r • 








TABLE 9 



Number of StatesHiat on the Average Consider 
1-4, 5-10, 11-25 or More than 25 Projects 
For Validation Each Year 



Table 9 reveals that there is a fair amount of variation among states in 
the average number of programs considered per year fqr validation. Of the 45 
states that have validation programs, one third (33X) consider 1-4 programs 
per vear, 42X (N«19) consider 5-10 programs per year for validation, and 18% 
(N«8} consldisr H-25 programs. Only two states, California and Texas, 
consider 25 or more T)rograms per year fgr validation; 'both of these states 
consider between 80 and lOO^programs each year. 



1 



X 

* ' K 



i 

V 



^ • - .43 59 



:RIC • 



Table 10. . Number of Programs and Practices Validated Annually (Average) 



Region 



Not 
Available 



1-4 



5-10 



11-25 




CEMREL (8) 

McREL (6) 

NEREX (7) 
NWREL (5) 
RBS (4) 
SEOL (4) ' 
SWRL (3) 



1 (AZ) 



6 (GA,NC,SC, 1 (FL) 1 (KY) 

TN,VA,WV) 

6 (IA,IN,MO, 1 (IL) 1 (MI) 

MN,OH,WJ) 

6 .(CO,KS,NB 
ND,SD,WY) 

3 (ME,RI,VT) 2 (MA,NH) 2 (CT,NY) 

3 (ID,MT,OR) 2 (AK,WA)' 

.2fOE,MI>) 1 (NJ) 1 (PA) 

1 CAR) - 2 (NM,OK) 
1 (UT)- „ ■ 1 (CA) 



1 (TX-50) - 



National 

T9;tals . ^ /I 

Percentage 
of Total 
■(N»45) 



28 
62 



^ 9 

0 

20 



.6 
13 



1 

, 2 



r 



44 60 



■- o 



• , TABLE 10 
) . 

Nurnber of Programs and Practices 
Validated Annually (Averagel 



Twenty-eight states {62% of phe 45 which have a validation process) > 
validate between one and four programs/processes each year. Ortly one 
state~Texas~validates more than 25 programs each year. 

Nine states (20%) validate between five and ten programs each year. Of 
the six states (1.3X) which validate'b^ween 11 and 25 programs annually, four 
average 12 programs, one state— ai if om+a— validates 25, and one ^ 
state— Kentucky—reports that the range is between 10 and 20. 




f 




ERIC 



45 



61 





0 

\ 


4 


• 








Table 11. Number of States. Usina Specific Dissemination Procedures . 


< 


1 

Region 


* 

Catalog 


~ t * ^ 

Brochures/ Resource 
Newsletters File . 


^ AMareness 

Conferences .Facilitators 
(Educ. Fairs) (Linkers)' 


e 

On-site 
. Open 
House 


Other 


AEL (8) 


FL.SC, 
VA 


FL^GA.TH, 6A,RY 

wv. , • ; 


GA,HC,TN,HV FL.NC 


NC.SC.VA 


* ELjKYfNCj 
SC.TH.VA 


CEWEL (8) 

* 


. IL.IN. 
MI»HN» 

iio Hi! 


IA,IL,IN, ^IL ' 
MI,HN,NO, ) 
OH, MI 


IA,IL,tN. lA.IL.IN.MI. 
HI.MN.HO, HN.m.OH.WI 
OH.WI 


lA.HO 




HcREL (6) 


NO,SD 


CO.NB.ND, 
uv 


HY CO.KS.MY 


NB 


SD.HY 


NEREX (7) • 


NH.NY. 
VT 


CT.HE.NH. MA.NH.'MY 
NY.RI.VT 


CT.ME.MH, CT.MA.NY.RI 
HY.VT , 






NMIEL (5) 

0 


OR 


AK^fD.MT, 'AK.OR 
OR.WA 


AK.HT.WA HA 




AK.MT. 
OR.tM 


RBS (4) > 


DE.TO, 
PA 

5. 


OE.MD.NJ, DE^MD.HJ.PA 
PA' 


DE.NJ.PAl oe.md.nj.pa 

r 


PA 




SE0L''(4) 


AR.HH. 


AR^NHyOKf NN9TX 
TX 


AR.HM.TX * 




TX 


SWRL (3) ' 


UT.AZ 

1 


AZ 

V 


. CA ' CA.' 




AZ.CA.UT 


National 
Totals 


26 




36 T4 


28 23 


7 ^ 


16 


NOTE: Nui^ers totat more than nuater of states because states do more than one thing. 








« 


* 


• 62 


* 
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TABLE 11 



Number of States Osinq Specific 
Dissemination Procedures _ 



"^Generally speaking, states disseminate information about validated 
programs through print media and through' "face-to-face" interactions, ."Print 
media* includes brochures, catalogues, and newsletters, which provide 
. descriptions 'of validated projects to other educators who may find the program 
to be of interest. Twenty-six states (58% of tKose reporting they have 
validation procedures) publish such catalogs and 36 J^80%) prepare and 
disseminate bfochures and newsletters, , 7— 

Resource files are maintained by 14 st^s-{315C), - Interestingly, two 
states which do nol-have program validation procedures have established 
"promising practices" files. Resource files typically provide program 
.abstracts and may include instructional materials developed by validated . 
programs as well as human resource listings, , 

"Face-to-face" interactions through which the states disseminate \ 
information about validated programs include awarlness^ conferences, education 
fairs, presentations at professional conferences and workshops, and on-stte 
"open hous^," In addition, state departments of education in 23 states (51% 
of those sMtes which validate programs) use staff or facilitators as 
"linkers,"^ These people bring togethfer educators with apartidular need with 
other educators who have developed a validated program to addriss^ a similar 
nee^, ' 1 ^ 

Other dissemination procedures include publishing artteles in educational 
journals, use of the meBia (radio, television, and newspapers), and traveling 
seminars. 

Because many states employ more than one dissemination strategy, nuntoers 
on this table total more than the 45 states responding to this question. 



\ ■ 
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ERIC 



47 



63 



Table 12, Responsibilities of the Developers of 
Validated Programs and Profccts ' 



Reglop 



>» 



o 



s 

0) 

a. 
o 



Si 

CM 



1. 



<0 



2 



2 



> 

40 



e 

£ 
S 

40 



<0 

a. 



CL 



% 



e 
« 

a. 
o 

> 



s 

a. 

T3 



a. 
o 



Q 



> 

2 



(A 
i/i 

(A 

p 



a. 



e 
o 



a. ' 



(A 

e 

.o 



a. 
o 



3 



> 



u 



AEL. (8) 
CENREL (3) 

HcREL (6) 
NEREX (7) 

NWRa (5) 

RBS (4) 
SEOL. (4) ■ 
SWRl (3) ' 



SC.TN 



lA.IL 
IN.NI 
MN.MO 

WI 1 



NB.KS 
SD 



CT.MA 
ME.NH 
NY.RI 

VT' 

ID 



DE.MO 
NO, PA 

AR 



UT 



OK.NM 
TX 



CA 



lA.IN 
HI.WI 



CO.KS 
NB.ND 
SD.WY 

CT.MA 
ME.RI 
VT 



AK,MT 
WA 



DE.MO 
NO .PA 

AR.OK 
N«,TX 

CA.UT 



KY.NC 
WV 

WI 



NB 



SC.VA 
WV 

MI 



KS.WY 



NH 



AK.ID 
MT.OR 
WA 



DE.MD 
NJ.PA 



CT 



AK.MT 
OR.UA 



DE.HO 
NJ;PA 



AR, 
NM.TX 

AZ 



olTAR 



,0K 
NM.TX 

CA.UT 



FL.NC 
VA 

lA.IL 
IN.NI 
HN.NO 



KS.NB 
ND.SD 
WY 

MA.ME 
RI.VT 



AK.MT 
OR.WA 



DE.MD 
NO, PA 

AR.OK 
NM.TX 



FL.VA 



OH* 



ND.SD 



CO.KS 
NB.ND 
SD.WY 



MT.OR 



N»t4ona1- 
Totars 



17 



-28- 



-20-^ — Z]- 



♦Responsibilities of Ohio projects are. self-determined. Theyj 
are encouraged by SEA staff to seek JDRP approval. 
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' Table 12 . , 

- , - ^ _ - _ . 

Responsibilities of the Developers of _/ 
validated Programs and projects 

Almost all 'of jthe "states which have validation procedures r.eqiHre that 
developers of validated programs and proj5ct§ actively particip?J:e -Ws^ =3^> 
disseminatifig thei);>^pwgrams. (The only exception is Ohio, where va^ated% 
programs have no mandatory responsibilities. However, they areVencouraged by 
SEA s,taff to seek JDRP approval.) / 

States that 'use the IVD process require that ^the school superintendent ■ 
Certify that, if state or other funds are available, the validated project 
will serve as a s'tate or national demonstration site for- a period t)f at leas't 
one calendar year\ftftm the-date of notification ( Sharing Eductional Success , 
page 45). Seventeen of the states,.require that developers of va.lidated 
projects continue to operate as a demonstration site for one year, twelve 
states for two, years, and $lne state for three years. 

Twenty-eight states require developerVsQf validated projects to ' 
panicipate in awareness activities, twenty states require projects to respond 
to written requests for informatioir, and twenty-one reqOire projects to 
develop and disseminate. materials that describe the project. 

A majority of the* states (30) also require developers of validated 
projects to provide technical ass'istance to school distrTcts which decided to 
adopt or adapt the project. Ten of those states require continuing 
contact—the developer must monitor the progress of the adopted/adapter and 
.must evaluate the effectiveness of the adoption/adaptation. Michigan • 
Projects,. for example, assist adopters in the preparation of grant 
submissions, helff them with budgeting,- and assist them in the selection and 
training of staff. 

TWo states,, North Dakota and South Dakota, require developers of 
validated projects to submit their projects to"* JDRP for approval. 
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Table 13. Nuwber of States Which Support the Activities of Developers-of Validated Prograros< . 



Region 



Technical 
Atsi stance 



Additional 
Funding to be a 
, Denonstratlon 

^ Pr^ect 



JORP 
Assisted 



10. 

Title 
IV-C 



Source of Funds 

State 
Dlssenhnatlon 
Grants 

—4- — 



Range 



(8) 



CENREL (8) 

HcREL (6) 
HEREX (f) 

NUREL (5) 



V 1 (GA) 
^ 2 (IM.OH) 



7 (a.KY.NC.SC, 
TN.VA.WV) 

'8*(IA,IL.IN^,NI. 
, , MN.MO.OH.WI) 



3 (TH.VAJW) 

8*(IA,IL,IN,^ 
\MN,HO,OH, 



up to^ 25,000 
6-70,000 



1 (KS) 



1 fID) 



2 (ND.WY) 4 (CO,ND,SD,HY) 1 (SO) 

7 (GT.HAiME, 6 (CT.MA.HE.NH, 
NH.NY.RI, NY.RI) 
VT) J 

. 4 (AK.MT.OR.MA) 



3 (CO.NB.UY) 4 (CO.NB.SD, , 5-100,000 
NY) 

6 (CT,HA,HE. 
' NH.NY.RI) 



4 (AK.MI.ORy 2 (HT.OR) 
HA) 



5-80.000 
10-100.000 



RES (4) 


2 (HO.PA) 


2 (DE.NJ) 


2 (DE.NJ) 


1 (NJ) 


1 (NJ) 


5-30.000 


SEDL (4)' 


1 (TX) 




• 3 (Api^oo 




3 (AR.NN.QK) 


up to 10.000 


SWKL (3) 


1 (AZ) 


1 (UT) 


2 (UT.CA) 




2 (irr.QA) 


up to 70.000 


National 
Totals 




15 


36 




30 6 . 




Percentage 
of Total 
(N-45) 


13 


V 


80 


4 


67 13 


* 



V 



*8e1n9 funded to do awareness and/or training activities Is Included In the definition of "demonstration project.** Not 
all D/Os actually operate visitation sites as part of their funding agreement. 



A 
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» ' TABLE U f 

Number Of States Which Support the Activities 
, , s of. Developers of Validated Programs ~^ ^ 

^ Once a program has been validated it may be eligible for additional 
funding as a demonstration site, to develop training materials and strategies 
to hetp adopters install the program, or to disseminate information in other 
ways.. Six states (13% of the 45 states which have a validation process) do 
not support- developers o^ validated programs. ' - ^ 

In 36 states (805(), D/Ds arei ^iqible for additional funding .to operatte 
,as a demonstration project, dissemin^iting information about the program to 
other educators, conducting training, and assisting educators who adopt/adapt 
the validated pt*bgram for replication. Typically, these grants r^nge from 
$5,000 to $3O,0dO for two years. In 30 of these states (67%), ESEA Title IV-C 
money is used for this purpose. Other states use state dissemination grant 
funds or special categorical funds. 

Irt addjtion to direct monies 'for D/Ds. state Copter projects of1;en 
contract. for services with the D/D. It shbuTd be noted that validation does 
not automatically entail continuation funding. • ' 

Technical assistance, often in the 'form of workshops focusing on 
disseijii nation strategies, is offered D/Ds by 15 states (33%). 




Table 14, States Supporting Adoption/Adaptation Activities > 
and Nature of Support ^ 

- • " Grants ' ' 
faq^o" state Given? Source _ Duration Amount 

AEL (8) • GA TA only 

KY Yes IV-C 

FL Yes IV-C - 1 year $ 5.000 

NC' Yes IV-C 

} SC Yes * IV-C 

TN Yes IV-C 

VA . Yes • IV-C 1 year $10,000 

WV Yes IV-C 18 months $ 5,000 

CO^IEL (8) • lA ' Yes IV-C . up to $5,000 

• 1-1.5 FTE for 
LEA training 
activities* 
up to $5,000 
up to $5,000 
up to $10,000 
up to $5,000 
varies 
$3-25,000 

McftEL (6) . CO . Yes . ' IV-C approx. $5,000 

up to 3K 
approx. $2,500 
approx. $6,000- 
, $5-7,500 



lA ' 


Y^s 




IL 


Yes 


tv-r 


•^IN 


Yes 


0 

IV-G 


HI 


Yes 


IV-C 


HN 


Yes 


IV-C 


MO 


. Yes 


IV-C 


OH 


Yes 


IV-C 


WI 


Yes 


IV-C 


CO 


. Yes 


- IV-C 


KS 


Yes 




NB 


Yes. 


IV-C^ 


NO 


Yes 


IV-C 


SO 


■ Yes ■ 




WY 


No 




cr 


Yes 


Mv-c 




Yes 


IV-C 




YeS 


IV-C 


NH 


Yes 


IV-C 


flY 


Yes 


IV-C 




Yes • 


IV-C 


VT 


Yes 


IV-C 



NEREX (7) CT Yes Mv-C 1 year approx. $5,000 

1 year - approx* $3,000 

1 year approx. $7,000 

1 year approx. $3-6,000 
approx. $7,5d0 

1 year $6-10,000 

1 year approx»$5-6,000 

NWREL (5) , AK I No Data 

ID r Yes ' IV-C '-$800-7,^00 
MT Yes , IV-C 

OR Yes IV-C $5,000. 

^ ^ HA Yes ^ ' IV-C, State, IV-C, 

Career Ed. * $5,000,000;' 

, state, $1.2 

.IV-B . . '^-^^-mllHon 

$5-6,000 

up to $10,000 
{avg.$7-8,000) 
Planning stage 
avg. $5.-6,000 

varies 
varies 

total of $1.5 ^ 
. . ' million In ^ 
1980-81 

SWRL'(3), „ . AZ ' TA only ^ ^ - ' 

CA Yes ^ IV-C 1 year ' approx. $10,000 
''"^ . ^ (some NDM 
* fondsf 

. ^ tJT Yes IV-C ^ ^ varies ^ 



RBS (4) 


' OE ' 


Yas'^^ 


State * 


1 year 








Staff 




NJ 


Yes ' 


IV-C 


1 year 




MD ' 


Yes • 








PA 


Yes 


IV-C 


T year 


SEDL (4) 


AR 


' Yes . 


IV-C 






NH 


No 








OK 


nres 


IV-C 








Yes 


IV-C 





\ 
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TABLE 14 



Number of -States Supporting Adoption/Adapt 
Activities and Nature of Support 



ation 



. The large majority of the states involved in th6 validation of exemplary 
prQgrams-provide support for. adoption/adaptation (A/A) activities. 
Th1rtv-f1ve of the states reported that they award IV-C A/A grants ranging 
' from $500 tp $20,000, with the average being about $5,000 to $8,000, to assist 
district? with the implementation of, validated projects. In most cases the 
grants are for the first year. Most states also stipulate that the monies are 
to be spent bn purchasing project materials and for staff development. The 
monies are not intended to support new, staff. In the majority of states the 
grants are awarded on a competitive basis. In recent years, however, 
sufficient funding has been available to meet the needs of most A/A applicants. 
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Table 15. Number of States That Provide Some Form of 
Assi-stance for Non-Validated Programs , 



REGION 
AEl (8) 



YES 
7 



DESCRIPTION 



CEMREL (8) 



McREL (6) 
NEREX (7) 



"iWREL (5), 

\ 

SEDL (4)" 



SWRL (3) . 



FL 
GA 

TN 
VA 

lA 

IL^ 

MO 



TA Provided 
TA Provided 
TA Prpvlded 
TA Provided 
TA Provided 
TA Provided 
TA Provided 



by state education agency 
by state education agency 
by state education agency 
to projects with potential 
for projects with potential 
by state education' agency 
by state education agency . 



Helped one project In the last .3 years 
Provides TA for projects prior to 

validation consideration 
Provides Informal ^assistance 
.Helps on data reporting 



KS - TA, validation team offers suggestions 
NB - TA Prjjvlded by statte education agency 
NO - TA Provided by SDE^on request 



CT 
MA 
ME 
NH 

NY 

RI 
VT 

AK 

ID. 
MT 



TA Provided through State Facilitator 

TA. Provided by. state 'education agency 

TA Provided by state education agency 

TA Provided by state .education agency and 

State Facilitator 

TA Provided jsy state education agency 

TA Provided oy state education agency 

TA Provided by state education agency 

Mini-grants, TA provided by state education 

agency , < ' W ;, 

TA Provided byyState^on request 
TA Provided by'^state on request 



OE - TA Provided by state on request 
MD - JA Provided by state on request 
NJ - TA Provided by state on request ' , > 
PA - TA Provided by state oa request 

AR - TA Provided by state; projects can:.'also 

apply for a grant * 
OK - TA Provided by statS education agency ' 
NM - TA Provided by state education agency 
TX - TA Provided by state education agency 

AZ - TA Provided by state education agency 
CA - TA Provided by state educatton aaency on 
request 

UT> - TA for projects with Provisional 
EndorsemeTit 



National Total 



35 



54 



TABLE 15 



Number of States that Provide Some Form "of Assistance 
for Non -validated Programs 



Most of the states (35 or 78%) offer assistance to projects that are not 
-approved for validation, but they do so only if assistance is requested. 
Formal feedback to the developers about the weaknesses in the application for 
validation "and suggestions for correcting those weaknesses is standard 
procedure. In some cases, this is done during the pre-screening process; it 
may also be done by the on-site validation teams. Projects which. are not 
validated but which have high potential for validation may be given intensive 
assistance^ in resubmission of their application. In two states (North 
Carolina emd South Carolina) technical assistance is given only to projects^^ 
with high potential; in two states (Arizona and Colorado) projects whiph have 
potential can apply for additional funding;' and in one state (Utah) projects 
whi-ch are given provisional endorsement may apply for- funds to further develop 
and to demonstrate the project. 





Table 16* ^'um^er of States :That Require Revalidation of 
Validation Programs 












Region Yes 


No 


Has Re-validation Been Considered? • 




AEL (8) • 2* 
♦ 


6 


No (4) 

Yes. beina considered' 
Yes, but rejected (1) 




CEMREL (8) ' 

0 

' s 


8 


No, 5-year limitation on funding the ^ 
projects means they all run out before 

xupy u/niilH nPPH fr) hp v*p<»\/;*1 ^Ha+ori 
.uiicjr ffuu lu iiccu i#u uc rv^ vai lu^ucu 

anyway 




McREL (6) 


6 


No (3); in 1 state, team has to be 

available for revisit 
Yes (3); 1 state will re-validate after 

2 years 


• - 


NEREX (7) 


7 


Yes (2) 

Ki " uQucauiun cunGnQincnLS OT \j/o place a 
restriction on funding of demonstration 
activities beyond 5 years 

NY - npvplnnpr^ tktp rPniHrPri fn \/;iniri;^fP 

new components and new target groups for 
previously validated projects before , 
thev ran dpinonstratp thpm« 


• 


NWREL (5) 


4 

/ 


WA - Consfdering re-validation every 
. 3 years 




RBS (4) 


4 


NJ « will Start rp«val1dat1nn In IQfll 
PA - surveys programs each year 




SEDL (4) 2* 


1 


NM - considering removing validated 
programs from th*e file after 2 years; 
r: programs could then apply for 
; re-validation 




• 

SWRL (3): 

• 


3 


AZ - 2-year life on validations 
^UT - Endorsement is for" a specified 
period -of time. ^ 


« 

• 


*Georqia~each year each validated project is re-ivi sited by SEA staff to 
' assure that the^program is still effective arid that resources for awareness, 
TA, .and 'Staff developmentr are adequate, and to determine that people are 
observir^j and then adopting. ' / 
Kentucky—after 3 years 
Oklahoma . • • , - 
Texas—after 2 years • ' , • 


ERIC 






56 73 • 



TABLE 16 . ^« 



Number of States that Reqif'jre Revalidation 
' of Validated Projects • ' ~ 



Table 16 reveals that revalidation procedures have been established by 
only four states: Georgia, Kentucky, Oklahoma, and Texas. Reval^idation 
t)rocedures are being considered by six other states^ particularly tonralidate 
new components of validated projects and potential applications to new target 
groyps. Overall, given the fact that there is generally a five-year limit on 
IV-C project funding, the projects expire before revalidation is necessary. 



; 



0 B7 ■ 
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Table 17, Participation in Cooperative Activities With Nearby States 



Region 


No 


Yes 


Description 


AEL (8) ' 


i " 


8 (FL,GA,KY, 

NC9SC9TN9 


—Regional training of validators 
. —Use people fromother states^ as 
~~ ^rHdators— ^ — — - ^ _-_ _ 

—3 share catalogues and/or literature 


CEMREL (8) 


4 (IN.MI, 
OH.WI) 


4 (lA.IL^N, 
MO) 


—Use people from other states as 

validators 
—Regional "training of validators 


McREL (6) 


1 (CO) 


5 (KS.NB.NO, 
SD.WY) 


—Use people from other states as teani 
leaders 

—Use people from other states as - 

validators 
-^Regional training of validators' 


NEREX (7) 


• 


7 (CT.MA.ME. 
NH.NY.RI. 

vtI 


—Participation U USOE catalog and 

information exchange 
—Use people from other states as 

validators 


NHREL (5) 


4 (AK.ID, 
OR,WA) 


1 (MT) 


-Planning exchange with adjoining 
state 

•^Use' people from other states as 

validators 
—Considering exchange plan with USOE 
. Regional Office 


RBS (4)- 




4 (OE.MO.NJ.- 
PA) 


—Participated in conference to share 
information on validation and to 
discuss issues 


SEOL (4) 


1 (TX) 


3 (AR.NM.OK) 


—Exchange of validat^n team 

—Exchange of directories 

♦*iecnmcai assistance from validation 
coordinator (NM) to other stjttes in 
the process of developing a state 
validation plan 


SWRL (3) 


2 (AZ.CA) 


1 (UT) 


—Exchange of validation team members 


National Totals ' 


12 


33 




Percentaae 
of Total 
(M-45) 


27 
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TABLE 17 



Participation in Cooperative Activities ^ 
• With Nearby States 

'. ** 

Thirty-three states (7356 of the 45 states with validation procedures) 
report that they are involved in cooperative activities with other, nearby* 
states. Cooperation occurs in the areas. of validator' training (often ■ 
organized by the ROEPK use of out-of-state teams, and_exchange of 
JnfQrmation, usually through program catalogues but also through conferences 
and worlcsfiopsi^ On&-state~l<ew^4fexicow^reports that the validation coordinator 
has given technical assistance to other states which are in the process of 
developing state validation plans. / 

J Several states report that the use of out-bf-State validation teams, has 
declined or will probably decline as the Impact of reduced ESEA IV-C funds 
begins to be felt among the states. 



> 



CONCERNS AND RECOMMENDATIONS 
I. ' 

One of the goals of the Validation Task Force in conducting the survey 
was to identify concerns and. issues about the validation of educational 
progra m s and pr a c tices which state -contacts feel should be brought to the 
attention of federal agencies which sponsor the development of validation/ 
procedures. This section reports thei.r responses. 

Many respondents felt that"the-revised-IVD- process was„ meeting their 
needs adequately. Frequently ttiese respondents characterized JDRP as "too 
rigorous" and" "inflexible." A common criticism .of JDRP, which tias been 
overcome in the IVD, was that. a site visit was not a mandated feature of 
the JDRP valfdeLtion process. ' ' 

Some respondents mentioned that JDRP seems more interested in the 
research aspects of a pro'gram than tn its 4)otential for improving education. 
On the other hand, one respondent criticized JDRP for, relying too heavily 
on classical research/statistical methodologies and felt that newer methods 
could be used by JDRP. Causal path analysis and regression analysis were 
cited as two such.newer methods.- " ^ , 

However, the IVD was not without its critics. Several respondents 
criticized the "sketchy" nature of the IVD training "and the multitude of 
state procedures. „ It wa? felt that IVD has little meaning outside of the 
state validating. a given program. In short, while JDRP is criticized for 
being too rigorous, IVD is criti'cized for being "watered down" and having 
no real standards. 
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Other concerns which surfaced included issues focusing on funding. 
Concern was expressed that D/Ds are not accountable for their work, either 
with adopters or for new development. It was felt that funding to permit 
closer monitoring of D/Ds could alleviate these problems. Several 
respondents felt that smaller states were'especially hurt by IV-C funding 
cuts, particularly as more LEAs in the states become aware of the value 
of validated projects. The reduction of funds to both D/Ds and adopters 
will have ajjegative impact,, just as the process and, its value is becoming - 
clear to some LEAs. 

_ _ _ ff^ 

Finally, concern was voiced that national standards^ result in 

irrelevance to some pop^'ulation groups. Bilingual and Native American 

populations were cited as examples of poflulations f or <f«^om many IVD 

questions are irrelevant and conversely whose programs were not evaluated 

' \ • I ^ 

by criteria meaningful to the program developers. ' 

< « ■ " L 

In additfion to voicing concerns, respondents- made a number of positive 

recommendations for strengthening the process of validation and disseminatio 
of programs. ~ ^ 

Coordination and communication were^the foci of several recomnendations 

• There should be more and better communication with' federal 
officials in the regional offices. \ 

• Funding should be made available to encourage contiguous 
states to work" together for validating programs. 

• The encouragement of use of one set of criteria by all 
states would- permit sufficient reliability to permit ^ 
easier adoption across state lines. 

• IVD criteria should be accepted by all ESEA categarical 
programs. . • ' 

• State/regional/national catalogs' of programs should 
include data about use. of projects by adopters. 

3eyeral suggestions concerned the scope of programs examined by JDRP: 

• JDRP should look at programs in career education, nutrition 
education, child development, etc. . -s. • 
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• JDRP sh'ould examine higher education programs. 

• JDRP should examine programs developed in non-public schools. 

• JDRP should encourage programs to submit other than just 
those that are student achievement-oriented. 

JDRP s.hould examine innovations of a 'program developed by 

adopters. * 

X 

Two suggestions which were mentioned frequently concerned JDRP procedures 
and standards. ^ 

• JDRP should mandate site visits. 

• JDRP standards should be more flexible. 

Several recommendations concerned funding: 

• D/Ds should receive more support for ongoing development ^ 
activtties. 

• Funds should be dedicated to encouraging' regional activities. 

• The five-year funding limi4^should^be_re-examjlned. 

• IV-C funding should be inx; -eased as more LEAs become in valved^ 

A number of recommendations cjntered on knowledge of diffusion: 

• Catalogs should be updated and non-functioning programs 
eliminated. NDN files sho^ild also be updated. 

• Adopters should be followed up in a systematic way. ^ 

• Re-training shiould be provided for adopters periodically. 




" APPENDIX A: STATE SUMMARIES • 




ALABAMA 



Prior. to 1976, Alabama used the IVD process to vali'date educational 

• programs and practices. This system is no longer in use. However, the 
Alabama Information and Development System (AIDS) maintains a resource 
file which includes Alabama promising programs and'practjce? (nominated 
by local superintendents and SEA consultants), NDN exemplary programs,- 

* and Aluama SDE documents . 

Selected abstracts from this file are disseminated to educators who 
request infbrmation on a given topic. - ^ 



Contact Person 



Dr. R. Meade Guy ^ 
^Director, Alabama Information and 

Development System * 
Room 607, Staft^ Office Building 
Montgomery, Alabama 36130 

(205) 832-3138 
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, ALASKA 

Validation Process and '.Crlteria^ Used . v, ' ' (■' 

Alaska makes use of a state-developed procedure, which is a variation 
of the IVD process. "Criteria fqr Excellence" have been developed 
jointly by practit^pners and Department of Education (DOE) staff- iji the 
following areas: reading (K-12), me^thematics, health, bilingu&l/ 
bicultural, gifted and talented, staff development, certified staff 
evaluation, guidance/counsel-ing, and corrgspqndence study. These ^ 
criteria listings are, available on microfiche^pm the San Mateo 
Education Resources Center (SMERC) and in hard copy from the Alaska DOE. 

, -' - ■ 

« ■ 

Training Procedures for Validators ^ „ 

» ' 

Using the Criteria for Excellence, DOE staff membfers conduct one- to 
two-hour qrientation sessions, during, which te^i members are taken , ■ 
through the validation process. Alaska has approximately 100 trained 
validators. . ^ " 

Validation Procedures , . » ^ 

The validation procedure is conducted by in-state validation JteSms 
composed of a content specialist practitioner, _a\building principal, a 
-public relations person, and a team leader who Ys usually someone from 
the DOE. In-state personnel are utilized iiv ordep to conserve financial 
resources. Site visits, usually one day in duration, are conducted by 
teams of Ihree to five people. . * ' 



Organizational Arrangements . , ^ 

One DOE staff member^''seyves as leader of the state validation effort, 
with other staff<participatnng as requested. Between five and twelve 
prpgrams/pr act ices are considered for validation each year, and an 
average of six are val4cte"^ed apnually. Those failing to meet the state's 
validation criteria may receive help in the form of technical assistance 
and/or mini -grants from the state. 



Support and Dissemination • . 

Support is pr'ovided to developers of validated programs and projects 
via Basic Skills Improvement and ESEA Title IV-C grants which total 
$50, 000-$ 100, 000. Applicants compete for grant awards, but promising 
practices nominees are* given priority*' The grants are-for one year and 
range between $1,000 and $5,0Q0. 
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ALASKA (continued) 

•A variety of kinds of support are provided by the state, .including 
ass+sting projects to locate consultants through the state's Talent Bank, 
providing release time for staff development, supplying self-assessment 
packages to, projects, and disseminating information about validated 
programs. .Methods of information dissenii nation include: press releases 
to all iiiedia statewide; ^articles .in the monthly DOE publicati(Jn Alaska 
' Education News ; ?w?irding certificates of merit; entering abstracts of 
^ validated programs/projects into the computerized Alaska Knowledge Base; 
and providing, bookmarks (conference handouts-)'' lijsting all promising'- 
practices, developers are also invitecl to present their programs/ 
projects fit appropriate conferences throughout the school year. 

Developers of validated programs and projects are not required to ' ' 
joperate those projects for any specific length of time. However, the 

5fr^>rogram(5 and projects are validated for three .years, at which time the^ 
may renominate 6nlJ do anothei?« self -assessment. They may update thefir 

• abstract and* renew' if they care to. They are required to provide ' 
technical assistance, develop and -disseminate materials, and mak-e 
thdnselves available to respond to requests*, from adopter /adapters^ - 

*.Ci&/As)^ Th.ey need not conduct training ^at their own or the A/A site, nor 
are they required to monitor or evaluate A/A activities^ They may take 
the steps -^necessary to prepare *f or JDRP, but-ijhey are no't required to do 
so. I. • ft • • . 



Contact Person ' • . , - 

Eula Rtiby, Director * . • ' ' 

Divisfbn of EducatiiDn. Program Sut)port 
Alaska State Department of Educart:-ion , 
State Office' Building, Pouch F- 
June^u, Alaska,' 99iBll ; . 

(907) 465-^830 ^ ' • , ^ ' . . 



ARIZONA 



Validation Process and Criteria Used 

Arizona is, developing its own validation process. The process^is 
presently in the development/pilot stage. Programs are rated according 
to four general categories: instructional objectives and program • ^ 
delivery; evaluation and results; setting, resources, and exportability; 
and management staffing. 

Training Procedures for Validators 

Validators are trained through a one-da'y workshop conducted by 
Arizona State Department of Education personnel. 



Validation Procedures • ' 

The Verification Team is composed of in-state persons who are 
independently appointed and assigned by the Arizonji Department of 
Education to^erify a program. During pilot stage, team members are' ADE , 
personnel who^are professional educators and/or contents, program, or 
evaluation specialists. One-day on-site visits are conducted by' teams of 
three or four people. ^ " , 



Organ izatipnal Arrangements 

Because the process is now in the pilot stage, the staff for the 
validation progranf has-not been assembled. There is no information 
available about tHe number of programs which will be validated each 
year.* The number Ipf programs which will be re^fTewed will be determined 
by the number which apply and are accepted by the review coipittee to be 
established by the Department of Education. Projects which are not 
validated can recfuest professional technical assistance as provided by 
program units in the Arizona Department of Education. - ' . ' 



Support and Dissemination v 

~ \ * ' ^ 

No/unds are currently allaeated to support the activities of 

(jevelopers of validated "programs.. Adaptation/adoption activities will be 

supported' only through ndn-financial. technical assistance. 

• Information tabout validated programs will be put into. the state's^ 
data retrieval* system. jA catalog of abstracts. will be circulated to 
local- education Sgencies and tither interested persons. In addition,^ 
Appropriate announcenjents will be directed at staff in local education 



ARIZONA (continued) " ^\ 



a^ncies;through various forms of media. ^ Responsibilities' of the : 
" developers of validated programs will include responding to inquiries and' 
sharing' information wjth potential ^adopters. — ^ 



CortJ 



itact Person 



Dr. Beverly ^Wheeler 
Di rec tor of nf ormat i on Di ssemi n a^i on 
Ar-izonaTtate Department of Education 
1535 West Jefferson 

Phoenix, .Arizona 85007 ^' 
(602) 25B-5235 





ARKANSAS 



Validation Process and Criteria Used v. 

Arkansas uses the. IVD process and^,thg validation criteria'' outlined in 
the Sharing Educational Success manual. 

Training Procedures^ for Vafidators % ' ^ 

Validation teams are trained in the IVD^ process, by 'trainers certified 
by the ll/S; Department of Education. -'^ " ' 

* * 

Validation Procedures. 



, The validation team consists of three people:* an experienced * 
validator to evaluate effectiveness/success; a person experienced In 
public school finance; and a person experienced in curriculum development 
.and program exportability. Out-of-state teams are used in order to ^ 
achieve a totally objective review free of- in-state influences. The team- 
conducts a three-day site visit. - ' 

Organizational Arrangements ^ * 

The Title IV off ice* has one person who ts* responsible for coordi- 
nating validation activtJties. Two or three programs are considered for 
v,alidation each year; on the average, twrf are validated. ' ' 

Projects can .submit an application for vskHdation any time after tftey 
have completeW their final evaluation (usually in the spring). If the 
application 4s acceptable, the validation team- is selected and a> site\ 
yi^Sit date is scheduled. , D/D application must be received by february/L; 
grants are awarded between 'March and June. 'Projects which are not " 
validated can, if eligible, receive additional funding dur,ing the - 
Competitive grant cycle and canMlso reqjiiest non-finafncJa1 technicaT 
assistance through thfe Title IV office. • ^ ' ^ ■ ' 



Support, and Dissemination ^ 

; Supfport of validated projects takes the form of .developer/ 
diemonst,rator -grants offered by Title 'IV-C. D/D- reipohsibilities are (-1) 
to maintain a demonstratipn site at the local district; (2) to provide" 
awareness activities upon request; (3) to provide staff development 
activities fgf adopters; and (4) to insure the availability of project- 
deyejloped nt^erials for adopting districts. Grafits are also made to 
districts wish-tng to adopt a state IVD validated prijject. 
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. , ARKANSAS (continued) 

^ • ■ ^ \ .' * . . 

^' ' ■ 

Information about validated programs is disseminated; through the 
Title IV project directory, an Ed Fair, state department publications, 
news releases, brochures, educational television, and official memoranda 
t6 a VI school, superintendents* * ' . ' ' 



Contact Person 

Mr.' Fay Bohannon • 

Coordinator, Title IV, ESEA 

ai vis ion of Federal Programs, Title IV 

AFCh Ford Education Building , . ' ' 

State Capitol Mall ', 

Little Rock, Arkansas 72201 

1(501) 371-1245 ■ i " 
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' ' CALIFORNIA 

Validation Process and Criteria Used 

California uses a state-developed process-for ESEA Title IV-C 
projects which consists of review of acceptable applications by a 
sevennnember review panel. A paper ^screening is conducted on each 
project submitted. Froimthe paper screening/ a number of projects are 
selected for representatives to be invited to present *^eir program and 
respond to the panel's questions. In caseB where the panel is impressed 
by a program's exportability but has so;tie questions about the student 
performance data presented, an on-site visit is conducted. 

Twelve. criteria are used in the validation processN 

1. Provides evidence that the original evaluation .design was 
credible and wel 1-mahaged. 

'2. Provides valid data which document that the original program 
produced desirab.le change in student performance. 

3. Addresses a statewide critical jieed which is clearly" 
/ ' ^ identified. 

\ 

4. Is economicaWy feasible—the dissemination/diffusion costs 
are reasonaple and the cost to adopters is feasible.' 

' r 

5. Incorporates^ jnstructional strategy for 
dissemination/diffusion which is considered inliovative, 
creative, unique, or original statewide. 

^ 6. Incorporates an instructional strategy and/or prodycjs which 
can bfe duplicated in a new location (replicability) . 

<^7. Incoirporates an' instructional strategy, and/or products which 
* . can be installed in pa^ts and/or phases (adaptability) • - 

8. Incorporates an Instructional strategy which is observable\ ( 
' anjj is clearly defined (communieability) . ' "7 

/ 9. Provides^ valid data that document the achievement of the 

/ ' process'^objectives of the original program. 

/ JO. Provides evidence that the instructional^, program and/or 

/ • products are ready for adoption. ^ ^ ' 

11. Provides assurance that the applicant is willing to serve as 
a dissemination/diffusion agent.^ 
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.CALIFORNIA (continued) 



12. Supports, complements, and supplements the local educ»ation - 
•agency's program in school improvement or master plan for 
special education. 

V 

Training Procedures fot Validators 

Deparl3nent of Education staff conduct a three^to four hour . 
orientation session prior to, the review process. *The sess,ion covers the 
responsibility of panel members, the criteria used for scoring, and 
appropriate interaction between the panel and pneserrters-. 

Valid ation Procedures* 

' f 

Review panels are formed to develop recommendations tcf the Department 
of education. rAll panel-members are in-state; each panel is corlposed of 

Five educators /rom outside tjie Department of Education 
w/lth expertise in content areas, ESEA Title IV-C, and 
issemination/diffusion strategies 




e/ community •member who has had extensive'experience with 
ducHion and spefcif ically witti^ESEA Tttl/e fll/IV-C programs 

6ne Oepartnent of Education Title IV-C staff member who had 
lirnited or no contact with the program dur4ng operational 
years. n ' 

All panel members ar^nn-state because the Department of Education feels 
that, the be-st resotfrces are in the state;, this proceotrre is also cost 
effective and effii^ient. A one or. two clay^sit? visit fs condjjcted by a 
consultant in the Of^ce of Program Evaluation and Research to review 
— aluation data, format, etc. 

Organizational Arrangements 

One full-time consultant is em^>1oyed by the Department of Education 
to coordinate validation activities fpr the state. In addition, eight 
regional. consultants devote 5 to 10 percent of their time to validation 
as requested by the full -time ^consultant. Approximately 80-100 programs 
are considered for validation each year. Of these, approximately 12 
programs are newly funded to disseminate for threevears. Twenty-five or 
njore are valfdated as exemplary and are funded to (Rsseminate each year.V 
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CALIFORNIA (continued) 



The schedule is as follows: 



First Friday of 
February 

April 1 



April -May 



June-July ^ 
June-August 



Throughout the 
year : 



One-page application submitted by Interested 
applicants. ^ , * 

Applicants submit a report on program effect 
on student performance and program 
exportability.' 

Review panel conducts paper screening, .has 
selected program rispresentatives. make 
presentations,' consults with Department of 
Education staff members, conducts on-site 
visits where appropriate. J >/ . 

Notification flfjjunding. 

Workshops on preparing brochures, awareness 
materials, packets establishing adoptiW 
adaption miniijiun^riteria, and other ' 
diffusion activitlysiin California. 

Regularly scheduled meetings of project 
directors to share problems, needs, and to 
provide mutual 'support. , 



Support and Dissemination 

The California Department of 'Education supports the activities of 
developers of validated programs through grant awards that average ^ 
$70,00Oi Local education agencies which are interested in adoption/^X^ 
adaptation activities may compete for up to $10,000 (ESEA Title IV^) to' 
assist in installation of a validated program. • Adoptions are also 
paftiaMy subsidized by National Diffusion Network funding.* -The state 
disseminatesnnformatioii. about validated programs through traveling 
seminars, regional minifsemin«s. Centers ^or Education Improvement 
(CCEI)— three regions ih CaltT^rnia--l inks with other Department of 
Education staff,, and professional conference^ The developers of 
validated programs are required to actively disseminate^ information about 
their program throughout the state, to provide' alJareness/information ,to 
CCETs and local education agencies,'^o participate in the main traveling 
Seminar, and to attend appropriate state. conferences of profess4onal 
organizations. I 



Note: There ^ire other validation procedures in use at the state 
department, ir/cluding one for business practices. However,. IV-C is the " 
rtajor validator in the state^ 
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CALIFORNIA (continued) 



Contact Person 

' Ira p. Barkman , ^ 
Coordinator, E?:emplary Programs 

Replication Unit 
Califorj^ia St ate ^Department of Education > 
721 Capitol Mall 
Sacramento, California 95814 

(916^) 322-6797 ' . . , - 
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^ ' ' COLORADO . ' 

Validation Process and Criteria Used v 

Colorado-uses the IVD. process and the validation criteria outlined in ' 
the Sharing Educational ^Sufccess manual. | 

Training Procedures .for Validators 

/Validators are trained by ROEP staff in accoj^^c^^th U.S. 

Validation Procedures 



Departnient of Education guidelines. / 




The validation te^ is conposed of twp people: an evaluation 
specialist^ who is university-based, and a local school administrator. 
Thus, for economic reasons^ validators are from in-state. . Site visits - 
are not .part of the >ralidation p'r'bcedure. - 

Organizational Arrangements ./ * ' 

The Director of Title IV-C is .responsible for validation. Eight 
programs were considered last year, of which three were validated. Jbe 
process begins/'in July when applications of programs are screened. In 
i^gust, the validation team meets to discuss each accepted application*. 
T^ result of that meeting is v^tidation or rejection. .Rejected progi^amso 
can apply for additional IV-C^unding.^ - • ' 

Support and ylsseminatton ... 

Those IV-C programs' which^ are validated -can apply for state 
dissemination grants. These projects then launch a campaign to bring 
about awareness, training, and follow-up. The state also funds A/As 
throu^uV-C grants to rej^^licate validated programs. 

Contact Person ' ^ ^ 



Chock Beck) 

Director, Colorado State » 

FacUitator Project 
930 S. Lincoln 
Longmont, Colorado 80501 

(3031 772-442P 

- \ 



CONNECTICUT 



^Validation Process and Criteria Used 

Connecticut uses a state-developed process for the validatiorr of 
educational programs and practices. 

Projects or programs which .apply for validatix)n must provide evidence 
that each objective was accompli shM at or above the criterion level, 
that the accomplishment of the ob^«:tive was the result of project 
activities, and that the reported findings are educationally significant. 



ing Procedures for Validators 



gi • Connecticut has a panel of s^en people who review all applications 
Ifor validation. The panel memMr training consists of workshops on the , 
criteria used in the validatioif procedure. 



Validation Proce dures 

^ One-day site vis^^itS, conducted by a two- or three-member team, are 
part -of the va-lidation procedure. Jeam members are all from within the 
.state. One member of each team is/a qualified evalirator. Using in-state 
va^lidators means that validators will be easily accessible and that 
expenses will be kept at a minimum. In addition,/ the full panel of 
validators reviews all projects^ tliat have ^i^iea for validation and thus 
has a better understanding of the n^eds and possibilities. 

Organisational Arrangements 

Three State staff members participate in validation activities 
(FTE=»0.6). The State Facilitator coordinates the activities of- a 

'Validatfon Panel. Projects are initially screened by S'tate staff, and 
ofnly those which 'are most likely to be validafed are invited to apply. 
Of the seventeen projects. which were submitted during the first year ^ 
(1979-8P), thirteen received State Validation. Projects which are hot 

.validated are able to 'request technical assistance through the State 
Facilitator and are ^Sble to reapply during a later .cycle. 

' • ^ . ' / 

• Timeline .of the Connecticut Validai ion Program: 

December 1 > Letters of \ntent to apply are submitted 

. * * by LEAs \ • 

^January 2 ' ^ Approved projects are invited to apply 

' Tebryary 15 ' Applications for validation .are rec-eived 
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CONNECTICUT (continued) 



March 1 ^ 
MarcK-April 
May^ 
June 



Applications are submitted to Validation Panel 

Visitations by teams ' . 

Panel meets for final action on applications- 

State Board action and notification of 
applicants 



Support and Dissemination 

The state provides IV-C funds to validated developers in Jthe form of 
Demonstrator Grants. TheSe one-year grants during the fourth yeair of 
operation are usually less tttan $10,000. The funds are awarded for the 
following purposes: (1) continue the 'delivery of project services within the 
developer LEA, (2) develop dissemination and. train.ing materials, and (3) 
prqvide for the costs of awareness and train ing^ activities. J ^ 

^ ^ * '' ' ' 

The sfate also funds one-year Adopter Grant? ^fpr local educational 

agencies. ^ The lirt.it on these grants is $5,000. . Funds may bfe used to 

purchase training materials an4 services froi^^the developer and to provide 

stipends, release time and travel for the st^ff of the adopting district. 

Opportunities are provided three times % year to apply for Adopter^ Grants: 

January 15; April 15, and October 1.. ^ 



The developer/^reim)hstrator (D/0) of -a state-validated project 'receives, 
one year of funding* through 'a' Demonstrator Grant for the purpose of ^ 
dissertiinating its, practices state-wide. Aftjsr this one year these* 
demonstration activities xarf be maintained' by Adopter Grant funds which are 
used by adopter^ to purchase trainihq f^rom the>-0/D. Presently, there is no 
limit on the length of time that a'sfate-validated project may be considered 
.an official D/D. 

The State provides descriptions of all state-validated projects to all * 
local educational agencies. This is done in conjunction with the \ \ 
announcement of the^ Adopter Grant program each year. Moreover, during the ' 
year awareness conferences are held, daring which potential adopters receive 
informjition about state- validated model projects. • , 



Contact Person 

Dr. Roger f^chards^ 

Bureau of Research, Planning 

and Evaluation 
State Oepartnfent of Education 
Box 22T9 - 

Hartford, Connectieut 06115 
(20,3) 566-8250 



Dr. Richard E. Lappert 
.Coordinator, Title IV-B-C , 
State Department of Educatipn 
Box 2219 

Hartford, Connecticut "0.6115 . 



(203) 566-3593 
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DELAWARE 

Validation Process and Criteria Used 



.1. / 



Delaware -uses the IVD process and the criteria outlined in the 
Sharing Ed^fcational Success (SES) Manual. 


Training. Procedures for Validators 

Eleven personnel frorji the state of DeTaware participated in a two-day 
SES trairving session, invplving a validation submission simulation, 
sponsored by Regional Office of Educational Programs and state staff in 
March.W^. ^ ' n^' 

Validation Procedures ir • 

- 

Three to five member val idation teams conduct one day site visits of , 
projects applying for validation.^ The teams are drawn Ifrom the pool of 
seven state department staff (subject specialists, research staff , 'etc. 
and four non-state department staff (LEA administrators, higher^ 
education, etCf) who are certified validators. An out-of-state team 
chairperson is used to provide ah external flavor and reduce certain 
kinds of criterion. Normally," several things precede the visit:- ^ . 

1. State pepartment of Public* Instruction^ staff know about and 
— raoniiorjthe program - , - ^ * 

2. A proposal is submitted to the state 
« 

3.. Preliminary screening occurs— state staff help the districts 
det«ine if they are ready for a site visit 

^ 4, Approved proposals go to th,e selected lianel 

5* The panel decides to review or not to rjgview via a site visit. 

Organizational Arrangements * . < 

At. present, the state NDN Facilitator manages the state v^alidation. 
effort on about a 20 percent time basis. To date five projects have been ' 
considered for validation c^nd three were validated. The staters schedule 
for validation is as follow?: - . , \ ^ 

\ . January Announcement 

March Sdbmijsion and Screening 
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March 
April- 
May 
July 



DELAWARE (contihued) 

# 

Teams .selection , 
>;Site*visit , ° 
Approvals 

Additional funding for dissemination 



September—on • Additional data gathered for possible- JDRP 
ifublication. " • ^ 

^ '.Projects that'do not- pass validatiAn are encouraged to contract for 
evaluation expertise from sources of t^eirvown choice or from a State ^ 
University Evaluatioi\ Unit organized :to serve LEAs. ' 



Support and^ Dissemination 

Additional IV-C funding of $5,000 to $6,000 per project is provided- 
*to validateoTyojectfi (D/Os) for dissemination purposes only in the 
fourth year ofl the 'projects. The state assists local ^education agencies 
with $5,000-M,000 project ado|5tion (A/Ay gr^ants which' can be- used for 
staff develofwient and materials but not for hiring new p6r^ortnel. 
'Developer/demonstrators (D/Ds) must continue operation as demonstration 
sites, for at least one year, carry on^ awareness activities, depjpnstrate 
their prpject, and ass1$1>adoptprSj Information on validated projects\1^ 
disseminated primarily through the IV-C office and through the NDN State 
Facilitator's Offtce^ The state IV-C Preject Inventory Booklet , which 
'describes developmental and validated projects, is shared with ^all 
districts Annually. ' ' . ^ . * 



Contact Person 

Dr. Peter J, B^ichmajgn - ^ 

State. Specialist, Educational Facilitjitor 

and Program Dissemination. 
Delaware Department of Public Instruction 
P.O. Box 1402 , « 

Dover, DeTaware 19901 



(302) 736-4583 



* \ • FLORIDA 

Validation Process and Criteria Used 

The IVD process is used for validating educational programs and 
practices. In conjunction with it, the validati9n procedures of the 
Joint Dissemination and Review Panel are used.v-The criteria used are • 
those identified by IVD: evidence of educational change and ^ 
exportability. ' \ — 

Training Procedures for Validators 

A cadre of vaPf^dators/trainers was trained by ROEP IV staff. These 
trainers then train other validators who can be used in or out of the 
state. ' - . 



Validation Procedures ^ , 

; The validation team is- composed of three people: two evaluation ^ 
specialists and one exportability /content .specialist. Typically, one 
evaluation specialist lives in Florida while the other two team members 

N^aria from out-of-state. ^This mixture permits the team tb be cost 
effective and guarantees the independence of the team members. As part 
of the validation procedure, this th'ree person team spends 2, and 1/2 days 

"visiting the. project site. ' • • 

^ Organizational Arrangements^ ^ 

' One professional at the SEA has primary responsibility for 
.validation.. Whilef the number of programs considered for validatipn 
varies, an average of /ive each year is considered. In the past three 

years, all programs considered were validated. Since 1974 when the 

validation pVocess was initiated, 27 of 31 deyelopment projects have been 
validated. The other fou^. projects were not nominated for the validation 
process'. . • 

• r 

In March^of each year, projects, notify the SEA that they intend^, to ' 
sybmit a request for validation. In April, ^:he application is submitted 
by the praject for review and comments. The revised application is 
submitted to the State Advisory Council for review and approval. The 
project director attends their meeting to answer questions. Simul- 
taneously, the validation team members are identified. | 

In May, applicWions are forwarded to team members who schelJule site 
visfts for May and June. Following the visits,^ programs are notifiefl of 
approval and prepare demonstration plans. Programs which fail to be 
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FLORIDA (continlied) 



fail to be validated are provided technical assistance by the SEA so that 
they can be •strengthened, ' Finally, the SEA publishes a listing of all 
validated programs. 



Support andf Dissemination . * 

Validated programs are eligible for two years 'of demonstration. grants 
to share their program, in add^tiorr^he sta\e makes available adopter 
grants of $5000 for one year for the installation -and. evaluation of the 
validated^prosram. Duripg the period, the D/D assists the adopter in 
installing the program^ staff training^ and monitoring and evaluating the 
trial use of the ppgram. 

.The SEA disseminates information about validated programs through 
newsletters, catalogues, conferences, and the SEA dissemination network. 



Contact Person 

Dr. Ralph 6. Vedros 
Program Administrator 
Department of Education 
Knott Building 
Tallahassee, Florida 30301 

(904) 488-7128 



r 
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GEORGIA 

" • ' t 

Validation Process and Criteriej Used ' 
IVD procedures and criteria are used. 

Training Procedures for Valid^ rs 



Validators ar^ trailed by state department of ^educatidrr staff . Some 
team members, were 'trained byyptaff of the North Carolina Department of 
Public Instruction. / \ 



Valtdation Procedures * • > ^ 

Validation teams are composed of four people: a research/evaluation 
specialist, a content area specialist, an administrator, and the SEA team 
coordinator. All validation teams are composed of educators from outside 
of Georgia. This lends greater credence to team decisions' and also 
permits project .staff to learn about prpjects outside of Georgia. An 
on-site visit by the four member team of two-thr^e days is . required. 

Organizational Arrangements . . " ' 

Five FTE are responsible for validation of educational programs and 
practices. * Five or six programs are considered for validation each year, 
of which four or five are validated. . 



State Supporl^ and Dissemination * ^ • ^ 

Developers of validated programs are obligated to maintain 
observation sites and to assist other educators interested in using the 
project.^ Staff support funds apd materials production /unds are' 
available from- th^ state. Similarly, funds for adoption are available 
from the state. . ' 

The SEA, through the Education Information Center, maintains a 
validated practices file and produces and disseminates, pamphlets and 
awarenefss materials, and conducts conferences. Finallyy SEA staff visit 
each validated project annually to assure that the program continues to 
be effective and to ensure that materials (awareness, jA, staff / 
developmente) are adequate. 
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GEORGIA (continued) 



Contact: Person ^ 'r^ * 

Dr. Jess Pat Elliott ^ ' ' . 

Director of Researc+i 

State Department of Education 

Atlanta, Georgia 30334 i 

(404) 656-2402 



7 



^ J The draft of Hawaii's validation process is being prepared for formal 
adoption by the State Department of Education. J\t that point Hawaii will 
have its identification, validation, and dissemination system in place. 
A field test of the system will be conducted duri/fg 1981 -Q;^. 



Contact Persons 



Winona ChangAathleen Steffen 
Office of Instructional Services 
Hawaii State Department of Education ^ 
233 Soutli Vineyard Street 
« Honolulu, Hawaii 96813 

(808). 548-3425 ' " * , 



I 
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HAWAII 

# 

Hawaii does not have a formal validation process but does have a 
procedure for identifying promising practices. State, district, and 
school administrators identify projects according to criteria which 
indicate that the projects. are: . - 

• " «. 

- Effective * . " ' N 

' ^ . - Replicable ' ' ' \ 

- Congruent with policies and directions. . r 

Projects which meet these criteria become eligible for dissemination 
by the Office of Instructional Services of the, Hawaii Department of - s 
Education. * ^ ^ - 



^ . lOAHO • 

Validation Process and Criteria Used 

\ Idaho user the JDRR process (called National Validation) and an IVD 
variation (called State Validation) and the Idaho ^Commendation Model 
(called, Stat6 Commendation), The Idaljo Commendation is given to programs 
and projects which receive a favorable review.from an in-state, 
three-member review' team, and approval of that review is given by the 
State Superintendent '6T Public Instruction. .State Commendation qualifies 
the project for adopjtifoh within the state. In order to receive'State 
Commendation, developers submit an application detailing project • 
activities and costs and providing evidence -of project effectiveness. 
The accuracy and validity of the application are, then examined and 
verified by the on-^ite rfeview team, wh\kh rates the project for 
measurabillty of objectives, responsiven^s to a real educational need, 
transportability, adequacy of evaluation plan and instruments, and other 
variables. 

^- 

Training Procedure for Validators 

' Dr. Bert Stoneberg, iVi) and Dr. Roger Reynold^on have reviewed the 
IVD training and,: although this has not occurred as yet, they are 
equipped to train others. For the State Commendation process, some 45 
people are trained and participate regularly in r-eviewing projects. 
Approximately one-third of these individuals are membens of the State 
Advisory Counci 1, another third are from universities, and a third are 
from local scKool districts. - • ' 



Validation Procedures •) 

Idaho's validation teams are composed of a university professor," a 
local district person with project expertise, and -an advisory council 
Tuember.^ These "State Commendation process personnel are from within the 
state and conduct on-site visits of twq day's duration. 

Orqaaizational Arrangements \ 

Idaho has five Title IV-C staff members. One project was recommended 
for State Commendation in 1980 and in 1979 there were four. All were 
validated followlfig a procedui'fe which is incorporated into the technical 
assistance provided to each project during its three-yeir life. Both 
State Validatjon and ° State Commendation involve application by the 
project developer and on-site review by a validation team. Project 
success and exportability are the major criteria. Idaho does not 
systematically provide assistance to programs failing to receive 
validation; although assistance from Title IV-C staff is available on 
request. '.^ . 



92 



102 



^ IDAHO (continued) 

Support and Dissemination i _ 

Idaho does not make provision for continued funding to developers of 
validated projects. They are invited to use local school funds to meet 
project needs. Title IV-C grantSv for adopters/adapters ranging frqm $800 
to $7500 are awarded on a competitive' ba§is, with the average grant being 
about $3000. SEA staff monitor tFie use of project funds. , 

» ^ » 

Information about validated projects is disseminated via the SEA*s 
newspaper, memos to district superintendents, and other communications * 
sent by the SEA dissemination consultant and other SEA staff. 

'Developers of validated projects are asked to continue their project 
activities for one year beyond phase-out of Title IV-C funds. They are 
available to provide technical assistance and information, at the 
discretion of the district, and they develop and disseminate material*. 
Some developers monitor or evaluate A/A activities,^ but, preparation for 
ODRP is not part of their responsibility. 

Contact P6rson 
Charles Brown 

Coordinator of Dissemination 
Idaho State Department of Education 
Len B. Jordan Office Building 
Boise, Idaho 83720 ^ 

(208) '334-3225 ' . • ^ . ' 
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ILLINOIS • ' . 

Validat ion P rocess and Crit-eria Used - • 

nUnois uses the- IVD process and the validation criteria outlined in 
th^e Sharing Educational Success manual.. - 



Training Procedures for Validators 

One 5£A staff person is certified as a trainer and validator. In , 
November 1979, 20-25 people were trained in IVD effectiveness measures' 
and 20 were trained as exportability analysts. Ea<;h group received 1/2 
* day' of training, including" some cross-training. 



Val idation Procedures 

The vaUdation team consists of two people. A person with expertistf^ 
in evaluatfon works with a person with subject or administration 
expertise. (Especially for validation of special education projects, the 
content expertise is desirable. The team members are drawn from 
out-of-state,' but increasing costs may limit this procedure in the 
future. A two-day site visit is made by the two members of the team. 



{ Organizational Arrangements y ^ 

During the first and second ywrs of a project, the staff works to 
develop the project for \fiubmission for IVD. Typically, out of'ten or 
twelve programs^screenedi six ar;e nominated for validation. On. the 
average, Svix programs successfully undergo the IVD process and are 
validated each year through the- Title iv section* ^ 

' The timeline for validation is as follows:' 

Review of val idation"^ procedures with project 
' director. 

Notification of submission. 

Applications are due in the Title IV office, 
at which time they are screened by in-state 
team and Title IV staff. 

technical assistance provided for revising 
promising applications.* 

SEA notifies nominees; validation teams are 
selected and given project .applications. 



September 

October 
icember 




Q , January 

\ 



February 



ILLINOIS (continued) . 

• March ^ One day ♦regiop^^ ^meeting for all teams and 

nominated programs. ' 

April/May Oa-site visits. , ' 

June ^ Certification of* validated projects. 

^uly ; * Validated projects may receive D/D funding. 

September , Distribution of Wings of Progress ^ a catalogue of 
validated programs. 



Support and Pi ssemi nation 

Validated pipogrkms are eligible for demonstration funding. ' Funding 
levels vary across validated programs. Generally 1 or 1.5 FTE tof^provide 
in-service trainingLis supported by the state^ Adoptens are not funded 
direct*ly> although in-service training and materials are made available 
from the D/Ds. , 



Contact Person 

^Karol Richardson 
^Assistant Marjager^ Title IV 

Illinois' State Board of Education 

.100 Nq. First street 

Springfield, Illinois 267a8 ^ 

(217) 782-3810 
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INDIANA 



Validation Process and Criteria Used 

'. ' 

The IV-D process, and- IVD/JDRP criteria are used^ A state-developed 
process is used for in-state prograjns such as state history. ^ " 

Training Procedures for Validators - » ' ' . 

Validator training procedures are in accordance with the IVD prdgram. 

Validation Procedures ' . i r 

The guidelines of IVD are used .for building review teams. In-state 
validators are on "the. teams. Site visits of twp days are- conducted by. 
the team, which has ty*o or three members. ' ' ' 

Organizational Arrangements I * 

Six-staff members of tlje SEA are responsible for validation. -Each • 
year between one and four prbgraras are considered for validation, of 
which one or two programs are, ultimately validated. For programs faijiirig 
to be validated, continued technical assistance may be .available on a 
case-by-case basis. ^ . » ^ 

m - - \ 

Support and Dissemination , ' ' • 

SEA staff coordinate awareness activities and on-site trainiag 
provided by D/Ds. Up to $5,000 is available for assisting adopters, * in . 
addition to monitoring and technical assistance- l)y SEA staff All 
validated programs become part^of the in-state Adoption/Replication.- 
Grants Program. * 



.Contact 'Person 
Don Treibic 

Indiana Department of Public Instruction 
Division of Innovative Education 
Room*229, State House 
Indianapolis, Indiana 46^4 

(317) 927.6'386 > 
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^lOWA 

Validation Process -and Criteria Used 



THteT^D process and criteria and JDRP triteria are used. 

- ■ ■ ■-•'x ■ , •: / 

Training Procedures- for Validators 

. . A one-day training session in the I^D process- "is conducted, usually 
- on'-a regional, basis in conjunction with j^thec^states. Additional 
^personnel with ^perience/expeHlse are- recniited. 

^ Validatioh procedures ' " . / ^ . 

The Validation team ^is! composed of three peopTe:- the team leader, a 
content* specialist, and an evaluation specialist. -While both tn-state . 
and out-of-state specialists serve on th^ teams, it is. generally 

• preferable to have jll team members from <)ut-of-st ate. The team-visits 
the site being reviewed, spending two da^s with the project. 

Organizational Arrangements ^ • \ . • ' 

Responsibi'lity^for validationMs diffused through ttie^ SEA by funding 
.^source. To date, seven projects have been validated by IVD (the last one 
was in 1976), and eight projects hav^^e received JDRP approval in the last 

• five years. Increasingly^ the SEA Is emphasizing submission to JDRP. 
There is no spt schedule forJVD validatibfi. Ifhil/ IVD is part of the 

• state plan for ESEA Title IV| this is done"l)n an ad-hpc basis. In the- 
last three years, only one program failed to be va'^lidated after submission 

> for IVD review. Thus, no f^ormal mechanis?h*exists for assisting* programs 
failing v^al'idation. ' ^~ , • ' • ' ^ 

^ ^ ^ * 

Support iind Dissemination • ^ - 

A validated projgratt may qualify for fourth-year funding*" at approxi- 
mately its third-/ear level. ThyD/D i;5 responsible for maintaining a ^' 
demonstratibn* site^ and for training, adopter personneT^ as well as for 
• jn-state awareness. activities. " , • 
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Contact Persons 



Dayid Lidstrom, NDN State 
Iowa State Department of 

Public Instruction 
Grimes State Office Buildifig 
Des Moines, Iowa 5031^ 

(515) 281-3111 



IOWA (continued) 



acilitator 



Max Morrison, Chief 
ESEA Title IV and 

Dissemination Section 
Iowa State Department of 

Public Instruction 
Grimes State Office Building 
Des Moines, Iowa 50319 

(515) 281-5274 ' • 



ERIC* 
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KANSAS 



V 

Validation Process and Criteria Used 



Kansas uses the IVD'^ocess ami the criteria suggested in the Sharing 
Education al Success handbook. ^ ' 



Training Procedures for Validators 



/ 



Kansas has a team or pool of people who were tWned at USOE ^. 
sponsored training sessions in the early and mid-1970s. 

' validation Pfocedures ^ " . , ' 

Kansas conducts two day on-site visits as part of this validation 
procedure. Usually three to four evaluation specialists and control 
specialists are on the team. They use, one put-of -state person on the. 
team to add to the team's objectivity. .Only one out-of -stater is used 
.primarily for cost reasons. 

' Organizational Arrangements 

One state staff^member has primary responsibility for the'state's \ 
validation process. Usually, one or two programs are considered per^year 
for validation and one^oi* twcT are validated. 

The schedule for the validation programs in Kansas is as follows: 

' 1. Formal state invitation for projects in 
third year development to submit letter 
of intent. • December 1 

2. State conciirrance on "letters of intent." January 

3. Those accepted attend state session to-. 

prepare applications for validation. January 15 

'4. State review team reviews application 

for validation. March 



r 



5. Formal validation team goes on site visit. " May " 

6. If state validated and want to, help by October 1 
prepare projects for JDRP» 

« 

Programs that fail validation are generally left to tlieir own devices 
with regard to meeting the state requirements at some future dat^v 

' - \ . 
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KANSAS (continued) 



Support and Dissemination - 

The state does not directly support developerydemonstrator - (D/D^) 
activities, but statie IV-C monies' are available on a grant basis to 
issist adopter/adapters (A/ As) with the irtiplementation of state validated 
projects* The state duplicates. materials from validated. projects Vor 
interested school distritfts^ and us^es various other means for 
^disseminating information abbtit the projects. 

The state- requi rets that developers serve as demonstration sit|s and 
assist A/As wi-th implemetitation. • . 

Contact Person • 
Phil Thomas- ' ^ 

Director, Educational Improvement ^ . 

Section ' / , . . 

Kansas State Department -of 

Education 

120 East 10th - ^ » . 

Topeka, Kansas 66612 

(913) 296-3136 



/ 
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KENTl/CKY 



idat'ion-Process and Criteria Use(t 




A .sta:te-developed process is used.v^A-'field review team verifies that 
the prograffrsflreets criteria ofi program effectiveness," cost effectiveness, 
and exportabm^. For school business practices, the ASBO "Sharing 
Business Success^alidatiori program is used. 




Training. Procedures for VMidatofs 



Validator teams are trainecf U. S* Department of Education' 
certified"^alidators or by SEA staff. Training focuses on overview of 
the Validation Procedures program and use of the. Field Review Package. 

Validation Procedures 

Validation teams include one representative of the State Validation 
Review Panel, an SEA curriculum specialist for the subject area reader 
review, sn LEA or no«-SEA staff person' with responsibility in a program 
similar to the program being reviewed. 

^ 'Members of the validation team are all in-state educators. For 
"Sharing Business Succes.?," all validators are from out-of-state, as 
required by ASBO. 

Site visits of no more than- one day are part of, the validation 
process. These visits are made by all three team members. 

Organizational Arrangements 

Four Wsons from the SEA Dissemination Unit are responsible for 
program vamttion. In addition, 12 or more other SEA staff (from 
curriculum sections) participate in validation efforts. 

Validation is organized in. a three-phase mode.^ In the first phase 
(Important People - Important Programs [IP]) any program may be nominated 
for review. The person nominating the program provides data concerning 
the success and innovative nature of the program. These data are verified 
by SEA officials v<ho th'en include the program in a "promising practices" 
•file.^ Between 100 and 20*0 programs are included here. 



• The secqnd phase (10-20 programs) involves the field review of 
promising practices. Results of the field review are submitted to a 
State Review Panel, consisting of at" leasr 13"SE7^ staff. This panel, 
a 3/4's majority, votes to validate after studying field review 
recommendations. Finally, the panel recommends 1-5 programs for JDRP 
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KENTUCKY (continued) 



submission. Since this validation process is in its first year, no 
results have been obtained. SEA staff estimate that 10-20 programs will 
.receive state validation (Phase II) this year. 

No unusual mechanism exists for assisting programs failing to be 
validated, although technical ass-istanc^from SEA staff is available to 
them. 

Support and Diss emination 

i 

Support of D/Ds from the SEA' is decided on a case by case basis by 
the Bureau with which the program .works most closely. The validated 
program is eligible 'for funding from the SEA Bureau as a demonstration . 
site. Some support of adopters is available through Tit.le IV-C or ' 
through appropriate bureaus of the SEA. 

Moreover, the IP file is disseminated by the SEA once validated ■ 
programs respond to individual information requests. ^ 

The State Validation Review Panel grants validation to a- state 
program for a period of three years. . • . ^ 

^ * * 

Contact Person ' ^ * 



6eri Weaver 

Director, Division of Research 

and Evaluation 
Office of Research and Planning 
Kentucky Department of Education 
Capital Plaza Tower 
Frankfort, Kentucky 40601 

(502) 564-4743 ' 
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LOUISIANA 



Louisiana does, not as yet use a formal method for. validation of 
education programs and practices. The Louisiana Dissemination Netwqrk * 
(R&D) of the^SCA solicited ^lescriptidns of exemplary programs from local 
school superintendents for presentation at four region„al workshops held^ 
in the fall of 4980. A catalog of promising 'programs Jnd practices was i 
produced is a result of these regional\ workshops. During 1981-82, the J 
LDN{R&0) is pi anning'^to^ establish a State Validation process, which wiTi 
^b.g^Uot— test-ed^n^-^wo-ji^^ school' systems. ' • 

A State Facilitator is located in th^ State Department^ of Education 
and is part of the Title IV staff. The State Facilitator supports the^ 

4,efforts of NDM programs working in Louisiana and assists state-developed 
projects jn their JDRP submission. He also serves on the State 
Department of Education Dissemination Advi-sory Committee,^ in-house 

'advisory group to the L'DN(R&D), established as a result of the Louisiana 
Di-ssemi nation Capacity^^Building Project, jointly funded by the National 
Institute of Education and the Louisi^ana State Department of Education. 



Conta^ct Person 

Sue. F. Wilson v ■ ^ ' . 

^Section Chief, Dissemination 
Louisiana Dissemination .Network (R&D) « 
Bureau, of Development 
Louisiana'^State Department of Education 
P.O. Box 44054- ' • 

' Baton RougeV Louisiana 70804 

(504) .342-4268 or 4269 
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MAINE - 

Val idation Process and Criteria Used 

Maine uses aW IVD process with effectiveness and 
•export ability criteria. , , ^ 



Validators attdid national training workshops conducted by the 



Training. procedures for Validators 

Validators attwid na 
Department, of Education. 

Validation Procedure 

The validation teWn consists of two evaluation specialist's from the 
state university systAm who are experienced in the use of the national 
IVD process, a Department of Educat>ion consultant who specializes in the 
project's fields, and a Title IV staff professional. These people are 
airfronj within the stite. Maine originally used out-of-state validators 
but discontinued this aractice because of the expense involved. The 
validation procedure includes a oneVor two-day site' visit. 

I ♦ ■ * 

Organizational Arrangement 

Cbdrdination of validation activities, is handlecf'by two professionals 
in the. State department of oEducation.rJAn average of two or threie 
projects are validate each year. State^er^onnel are very selective 
with refspect to projects that are encouraged to apply for validation. ' 

If a, project is^ no.t* validated on the first try, it may request \ 
additional technical assistance from the state Title ly office andfmay 
reapply after completing the recommendations made by the vaTidation %eml 

Support and Dissemination 

Title IV-C projects which are validated by the State Department of 
Education can apply Hr fourth-, and fifth-year funding. In the fourth 
year, they can receive.up to -50 percent of the third-year funding, and in 
the fifth year up to 25 percent of the third-year funding. Schools which 
wish to adopt a project can apply for up to $7,500 for one year to adopt 
any state, regionally, or nationally validated project. 

The state disseminates information about validated projects and 
'proarams through a state directory, through cooperation with'the State 
Facilitator Center, and^ through RECON, the New England Regional 
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MAINE (continued) 



Communication Network catalogue. Projects which are validated are 
required to continue to function, within tfie local education agency and to 
serve as a demonstration site. The project dicector participates in 
workshops and in training sessions. i 



Contact Person ^ * * ^( 



Dr. Lois W. Jones ^ 
Title IV-C Coordinator 
Department of Educational and 

Cultural Services 
Station 23 

Augusta, Maine 04333 ^ 
(207) 289-2475 
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' ' MARYLAND 
Validaition Process -and Criteria Cf$'ed 

— \ ^ y- 

Mar)rland uses a state developed process to verify* Promising 
Educational Practices and Services (PEPS). The four major criteria for 
appr^pval by the PEPS verification program are: statistically significant 
project effects, effects judged to be the result of the project, 
educational significance of the project, and project transportability. 



Training Procedures for Validators 

At present, at least one person from each division or office in the ^ 
State Department of Education has been trained as a validator. Seven 
staff were trained at a Region III, ROEP, two-day "Sharing Educational 
Success" Validation Workshop in 1980. Five other state staff were 
trained via a one-day workshop conducted by PfPS core staff using SES 
(IVD) and state-developed materials. 

Validation Pf5cedures 

Jn Maryland a state verification team composed of at least two 
evaluation staff and two content area staff is. selected from the twelve 
available validators, after a program verification 'application is 
screened for completeness. The validation team then reviews the 
application for compliance witj^the PEPS criteria. If the project 
application is lacking in some way ^the* verification team can either 
request additional i(i-format ion or reject the application. \f it appears 
that the application satisfies the PEPS criteria, a one-half day site . . 
visit is scheduled. On site each member of the verification team 
completes a PEPS Verification Checklist. The bottom line of the 
checklist is approval or disapproval of the project based on the team's 
collective judgment. 

Organizational Arrangements 

One staff member from the Division of Instructiofr, PrograitNAss^smenf 
Branch, has primary responsibility ^part -time) for managing PEPS. Eleven 
other state staff at^e also involved 'on a part-time as-needed basis-. ^ 
Three of these staff are ^involved in IV-& work. 

The state verification process, PEPS, was initiated in the Spring of* 
1981 and is expected to be fully operational by September 1981. To date, 
four projects have been reviewed in a pilot test of the process ;"One was 
validated. 




^ *Maryland prefers to use the term "verification" instead of ^"validation" " ^ 

for the state process used to identify and certify promising educational 
practices. 



MARYLAND (continued) 



The state's verification schedule is as follows: 



Open Date 



September 15 and i 
^ March 15 



District: completes "PEPS Submittar • 
Fbrm" and sends it to Division of 
Instruction (DOI), Program Assessment 
Branch. 

, ^ „ J. 

The forth *is screened for cortipletenes^ 
by Program Assessment staff. 

A four or tivo^ person PEPS team or 
panel \s conv.pned to review the 
application for validation. _ 

The PEPS team re^iuests more 
informatioVi, rejects the application, 
or accepts the application and sets 
up a site visit. 

The PEPS team conduct^ a half -day 
onrsite visit and rates the program ^ 
on the Verification Checklist. "The 
program is accepted" or rejected' based 
on the team' jaicollect^ive ^ i 
recommendation. 

DOI and pi^oject staff meet to 
complete a^Process Resource Chart on 
the Project. " \ » 

- . ^ ..r - — 

Information about new^Pfomising 
Practices is distributed in" the PEPS 
manual. ' ^ '.A - 



State DOI staff plan to provide evaluation «assistance te applicants, 
on a request basis/ contingent on the status on validation* dpp^lication^ 
froffi'LEAs.. ^ ' ^ ^ 



SQpport and Disseminat i 6n 



^ At present the^tate <loes no£ provide financial support for 
dissemination 6r tfiTrtinued'development to the developers of verifie d ^ 
programs. Support is also unavailable to adopter/adaptiar (A/A) 
(|j strict s,- although the state hopes to support A/A activities depending 
(Sh the, futurfi/avatl ability of funds. ' 



ERLC 



107 



4ir 



MARYLAND (continued) 



-4 I 



Developers jnu St agree to, operate as a flenionstratid'n site for one year 
and to provided training and technical assistance;fb 'A/As. . 

State information about* verified programs Will be disseminated to all 
school principals twice annually through one' page 4dditions to- the PEPS 
fnanual. State linkers and school 'librarians ^111 ' assist with 
dissemination, . - ^ 



Contact Person 



John A. Johns 

Specialist in Program Assessment 
Maryland Sfate Department of Education 
200 West Baltimore- Street 
Baltimare, Maryland 21201 ^ 

(301) 659-2380 ^ 
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MASSACHUSETTS 



Validation Process and Criteria Used 

/ - ' 

Massachusetts uses,,a state-developed/ modified lYD process. ,The 
vali'dation pr^ocess J s accomplished by a one-day on-site visit by a 
validation team. At the, conclusion of the on-site visit, the program or 
practice *|ill either be validated as an exemplary model for state 
diffusiony^validafted for state information di^eminatjon, or not be 
validated. If the program or practice is validated for statewide 
diffusion, it will be eligible for financial- support from the Department 
of Educatiojl to share information about the program with-other school 
districts in' Massachusetts. 

> " ■ * ■ * " 

Validation focuses on three criteria: 

1. E vi dence of effect i venes >s . ^Supporting -evidfrnce is provided 
to Show that the attainment'of the major objective(s) can be 
attributed to the project activities*' 



ili^. 



2. Exportability . Information is provided to demonstrate that 
it is feasible to transport the program or, practice to other 
school districts apfd that it can be adopted or adapted by 
other school districts. ^ ^ 

3. Cost Analysis .. .Sufficient information is provided^ 
describing needed costs of, start-up, operation and * 
management,, and the population to be served, which, .when , 

bined with evidence df effectiveness and exportability, 
ssist an interested school district in making an . 
ed decision about adopl^ion or adaption of the program 
or practice. , ^ ' 




Training Procedures for Validators . ^ 

Validation tearts receive a full day/ of training from State Department 
of Education staff. Simulation materials have been prepared for use in - 
this training. Additionally, simulation training is provided for 
•yalidators of School Business^ Practices. A validator's handbook fqr^ 
Schopl Business Practices has been prepared, modified ffom the ABSO 
Validation Manual. An additional half-day^ of -training Is .scheduled for 
validators after they havie i?een gfiven- their site visitation assignments 
and are pireparing to review ja specific project or practice, ^oth 
validators and Title IV-C project directors attend a two-day seminar^^on 
.validation specifically aimed at the JDRP review process. Technical 
assistance is provided by the Evaluation ^Resource Center for those 
plrojects that intend to. apply for JDRP. validation.' * ^ . 
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\ MASSACHUSEHS (continued) 

Validation Procedures \ , 

• ' 

A three' person, in-state validation team, consisting of one, 
evaluation specialist and two practitioners, conducts a one-day sit6 
visit to review the project or program. Cost-effectiveness 
considerations have led .to the usevOf , in-state validators. The State^ 
^ trained ^a cadre of validators .over the years since the initiation of 

IVD. yM\)i of these persons were^previous (Title, III) project directors ^ 
who had projects reviewed through the IVD process as early as 1972. The 
validation process is managed by the Evaluation Resource Center ^taff, , 
^ * who prepare the schedule for on-site va^'idation and build the validation ^ 
teams. In the specific case* of school business practices, three school 
business officials (usually, at 1;he assistant suj^ferintendent level) 
comprise the teams. These teams are often v:omplemented by ^ . 
superintendents or Title IV project directors who sit on the team when 
;the practice concerns energy, school food services, or regional^ 
cooperative practices such as quantity purchasing. ^ ^ 

^ Organizational Arrangements * , 

Six regional program officers and three evaluation consultants are 
responsible for the implementation of the Massachusetts Validation 
Process.' Approximately 24 programs or projects are submittM for 
considferation each year. Df these, eight to ten are validated. 

Appulications for validation must be submitted to the Department of 
Education by April 1. Training for validators takes place in May, and 
on-site visits are conducted from May through September. Notification of 
approval is made in October, and a catalog descr.ibing validated programs \ 
^and projects is published in November. Projects which are not validated 
can requ^^st technical assistance from the State Department of Education. , 
Thisi^an take the fprm of evaluation support, assistance with the state 
mandated management system, or workshops to assist "projects in preparing 
for JDRP submission. ^ 

* ^ 

Support and. Dissemination 

^ The statff supports projects which have been validated by providing 

/ them with fourth-year funding. Districts which would like to adopt or 

adapt a validated project or program can apply to the state for a grant 

of up to $3,000. 

** •-» 

The state disseminates information about validated programs and ^ 
projects through newsljstters, awareness sessions, and MAGG, a 
.computerized" practice/file developed to be compatible with ERIC and the 
Bibliographic Retrieval SetvJce National Practice File. Additionally, 
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.MASSACHUSEnS (contlfiued) 

. I . - ' ' ■ 

the U.S.O.E. (Region I) diss^inates information on state validated 
practices through RECON, the New England RegionJil -Communice^tion Network. 
An index tp state-validated projects has been prepared by the RECON s^taff 
describing effectiveness, implementation requirements, financial 
^ requirements,! and available services. LEAs can then draw from the 
contiguous NeV(Jngland states when they decide to ado^Jt/adapt a practice. 



Contact Person. 
Dr. John Reynolds 

Massachusetts CoordinatoP, Title IV-C ^ 
Springfield Education Center , » 

155 Main Street - ' ■ . 

SpHn^field, Massachusetts 01105 

(413) 739-7271 / ' • ' 




MICHIGAN ^ 

' / ' * 

Validation Process and Criteria Used " , - 

"TT : / . . \ ^ 

Jhe Michigan Department of Education uses a state-deyeldoed validation 
process which tl described fully_lit^4^andbo5lr^veloped^^^^ i97X by the^ 
Department.' Criteria were estab^Hshed ToT'four project classifications: 
Planning, Da//elopmental /Operational, Experimental and Demonstration. 
Validation i^ required to move from Experimental, to Demonstration project 
'status. By definition, validation is the process of reviewing and, 
v,erifying the results ^of an evalliation of ^n experimental project. , - 
Projects are certified as validated by a Project CUssification* 
Committee, chaired by' the Deputy Superintendent, if the evaluation data^ 
show that they are toth effective and ^exportable. A major criterlon'for 
effectiveness is that the project meet- its specified performarKe, 
objective for specific target populations, Expor^bility is defined as 
replicability in the experimental year of'the project. OveraU, tlie > 
^entire project is reviewed inflight of .the^fol lowing questions to 
determine its qualifications for a "Demonstration" classif icatit)n: ' 

1. How accurate were the data ^mocessing procedures? 

2. .How accurately were 1;he data analyzed? 

, 3. What percent -of the conclusions stated in the evaluation 
report are supported 'by the .collected, data? 

4. What percent' of the process performance objectives are 
supported written documentation? 

f, ' " ' ^ ^ ^ . ^ 
ient is there evidence that informatibn gained . 
. from the-evaluatioh was utilized in management decisions? 

6. What percent of the product performance objectives were ^ 
\ successfully validated? 

Xjv--^ percent of the product performance objectiveis were 
^ achieved? 



Training Procedures for Validators ' ^ 

Research staff -of the Department train new validators to use the 
published handbook! Validators are" required to have a technical' 
background in evaluation and may not be currently employed by ±)ie 
Department or the school d-tjstrict operating the project to be validated. 
The oriefltation/training^ validators is usually accomplished in one day. 
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MICHIGAN (continued) . • , 

• • • 

Validat ion Procedures 

The state utijizes a pool of approximately 18-20 validators drawn on 
a voluntary basTs mostly from university settings in Michigan. The 
State-required one-half to one-day validation site visits are usually 
conducted by a one-person team. The rationale for using Independent 
evaluators as validators'is to avoid political bias (the validators 
receive honorariums) and to contribute to quality control. Validator 
on-site visitation responsibilities and procedures for data collection/ 
evaluation are outlined fully, with prescribed forms, tn the Handbook . 

Organizational Arrangements * ^ 

Michigan maintains approximately six staff pn the ^^perimental 
►Bemonstration Centers Program unit. One person manages the state 
validation activities. Although ttje number varies, approximately a dozen 
projects are considered for validation each year. On the average about 
10-12. j|?rojects are validated each year. Projects are removed after 
several years. Usually there Ts a mix of old and new Demonstration 
Projects. 

Validation can occur at any time of the year according to Michigan 
procedures. A prescribed sequence of events is^suggested, therefore, 
rather than specif icdates... * • 

1. The Sef vice Area staff select projects for nomination and 
validation. " ■: 

2. Th& Service Area se^lects a representative to serve as a 
facilkator for validation. 

3. The Service Area notifies the project staff of nomination. . 



Service Ar^a staff should check to determine if »the , 
superintendent knows and approves of the propose^ validation, 

4. The Service Area staff encuiflber the funds needed for their 
'total validation activity. 

5. The Experimental and Demonstration staff select validators 
and establish an on-site visit date(s). 

6. The on-site visitation Is conducted; 
7., The validation report is received. 
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MICHIGAN (continued) 



ERIC 



Support and. Dissemination 



^ Typically, the Department graats demonstration projects $60,000 to 
$/0,000 for one to three years for: travel, training, developing 
demonstration training materials, reproducing materials, and supporting 
the project director and a secretary. In 1980, about 25 projects 
received the above support. The Department provides adoptijDn grants up 
to $5,000 ($3,500-$4,000 average) for- A/A districts. Demonstration site 
staff alsb schedule follow-up visits to A/A sites to provide 
implementation assistance.^ 

Demonstration project staff are required to: conduct awareness 
workshops; develop and maintain "implementation checklists"; and helfr' 
client schools obtain adoption grants by providing assistance with grant 
forms, budget, training schedules, staff selection, implementation, etc. 

The Department employs dissemination procedures ranging from the 
preparation of jiwareness brochures and Department of Education Project^ 
listings 'to the conduct of state awareness conferences^ and subject area 
awareness workshops by RegionaJ Supplemental Centers. 

Contact Person , . - - 

John ds borne 

Coordinator, Experimental Demonstration 

Center Programs 
Box 30008 

Lansing, Michican 48909 ' • , • 

(517) 373-1806 
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' MINNESOTA 



'^ValidaMon Process and Criteria Used 

Minnesota uses~the current IVD validation process and the cfitfirtli 
cited in the Sharing Educational Success manual. 

Training P rocedures for Validation - , 

Minnesota draw^^pn a pool of people in the state who are trained ia 
the IVD procedure. 



Validation Procedures * 

Minnestota uses a combination of out-of-state and in-state people to 
pre-screen projects to, determine their readiness for a validation process 
site visit. If a project Is deemed ready for a site visit, one or two 

#in-state validators with expertise in evaluation and/or the content area 
conduct a two-day site review of the project. A combination of in- and 

^ out-of-state people Is lised to promote objectivity and as an expedient 
approach givenfcurrent resource problems. - - 

Organizational Arrangements 

, Minnesota use three full-time staff members from Title IV-C and two 
part-time people from the Office of Planning and Evaluation to conduct 
the validation "process. Typically six to nine projects- are considered 
for validation each year, and two to three are validated. 

k 

The state's schedule for validation is as follows: 

October First review of project status: mater ij^ls 



and evaluation data 



November-March Projects nominated for, site visits and 

teams assembled 

April -May On-site^validatid^f reviewj^ 

June . Reconvening of screening panels to review 

end-of-year data from projects recommended 
for validation. by on-site teams C_ 



July 1 , Funding of validated projects 7 

Basically, the state bas no formal structure to assist projects that 
fall short of the validation criteria. Project staff are free, to try 
again on their own efforts. 
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MINNESOTA (continued) 



Support and Dissemination 



(^i'\ 



Validated projects receive state IV-C funds to serve as demonstration 
sites (D/Os) and assist adopters. Districts choosing to adopt valida1;ed 
projects also receive- one yaar of IV-C funding to assist with start-up 
activities. ULfl^ staff are required to provide A/As with training and/' 
technical assistance. .The state disseminates information about validated 
projects' through the Title IV-C dtsaemination/awareness system. 



Contact Person 
Len Nachman 

Evaluation Supervisprx_ - 

Minnesota Depanfm€ntof~td'ucation 

Capitol Square Building 

550 Cedar Street' 

St. Paul, Minnesota 55101 

('612) 296-5032 , 
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MISSISSIPPI 



Mississippi does not haye a validation procedure at tKis time but is 
irv the process of developing a procedure. The proposed procedure will 
include pre-screeniog of applications by State Department of Education 
staff; review of the application by the validation team leader, an 
out-of-state person with U.S.O.E. certif icatio.n^iind five years of recent 
experience in education-aj management and evaluation; an on-site visit by 
the team leader and two additional in-state team members who have state 
certification, as validators; and submission of a report by the team to 
the State Dfepartment of education. Projects will be validated for their 
effectiveness/success (purpose ^and objectives, program activities, 
evaluation design, and Results and analysis); exportability; educational 
significance; target population; materials, equipment, and facilities; 
.and. staffing and training requirements. 

* 

Contact Person 
Clyde Hatten 

Coordinator— ESEA Title IV 
Mississippi State Department 
of Education 

P.O, Box 771 . _ , " 

Jackson, Mississippi 39205 

(601) 354-6953 ' " " ' 
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w MISSOURI 
Validation Process and Criteria Used 



Missouri Uses thf.- IVO process and the validation criteria outlined in 
the Sharing Edpcational Success manual. 

Training Procedures for Validators 

■ Missouri validation team members have received U.S. Department of 
Education certified tra4ning. The 20 or so validators on the state list 
receive a day of retraining each year in addition to pre-validation 
briefings.' * - - . . 



* Validation Procedures 

The, state usually conducts two-day validation site visits and uses 
three person'teams. The administrative teams- are selected to include 
administrative, evaluation, and subjec,t area personnel. .Out-of-state 
validation team members are^emliloyed to add to the objectivity/ 
credibility of the processT • ^ 



Organizational Arrangements 

I^l Missouri 3.5 PTEs are -on the staff of the division section ' V 

responsible for the validation of educational projects. Usually three or 
four projects, out of about 20 third year projects, are nominated,, per 
year for validation and most are validated. 

The st^te has no set schedule for valida^tion. Selected projects have 
$2,500 written in for validation expenses in their third year. Usually 
the state initiates a mid-year meeting to review project data to make 
"go" or "no-go" decisions regarding potential validation site visits. 

Projects that fail in their attempts at meeting the state validation 
requirements may obtain technical assistance on an infomal request basis 
from the Department -of Elementary and Secondary Education (DESE). The 
state, towever, does not have a fomal assistance program or procedure 
for providing aid to thlese districts. 



Support and Dissemination 

The state does not provide direct funding to assist validated 
'projects. Developer training activities, however, are funded through 



MISSOURI (continud«^) . • . x 

) •• • 

Title IV-C. The state does assist adopter/adapters (A/As) through the 
Pacesetter process as outlined in the IV-C ESEA Manual for Project" 
Applicants. 

# 

- Developers of validated, projects must agree to continue .project 
operations for one year, host visits by potential A/As, and make training 
available the summer following validation. 

jThe state disseminates information about validated projects through 
the Missouri Pacesetter catalog gind through handouts at DESE workshops. 



Vhe 
e>Hs 

e^a 



The^ate also disseminates informa~lion through the VIP Program and the 
Facilitator Project. f 



Contact Person 



Tom Odne?il 

Director, Title IV-C 

Missouri Department of Elementary 

and Secondary Education 
Box 480 • 

Jefferson City, Missouri 65101 
(304) 751-3468 
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MONTANA 



Validation Process and Criteri a Used ' 

*\ . 

In Montana a variation of the IVD^rocess is used, with some sections 
of the handbook^ having been rewrittjgn for clarity and to meet local 
.needs. The JDRP process is utilized when appropriate. Validation 
> fcr^erta reflect the concern that projects meet their stated objectives, 
• that they have a probability .of being useful and successful in other 
settings, that they be. cost effective, that they be "complete and intact" 
where originally developed' and implemented, and that they are worthy in ' 
the professional judgment of experienced educators and project validators. 

Training Procedures for Validators 

Professional educators from all areas of education received training 
in the first session conducted by the Title IV 'Coordinator at the Montana 
Of f ice .of Public Instruction (OPI) and, an on-site validation team. ^ These 
initial 15 trainees were taken through the validation process by the . 
group of experienced team members. There is an intent to conduct 
« additional training as needed. 

Validation Procedures 

ftontana validatiorf-teams are made up of an out-of-state person who is 
tralfffed in the validation process and who usually serves sis the 
chairperson, a specialist who is familiar with the content area of the ' 
predei^an administrator, and an evkluator.* On-site Visits of 2 or 2 
and Ifz day's duration are part of the valfbation' procedure'. Montana 
^educators involved in the project validation process feel that 
participation by an out-of-state person makes the process more objective ' 
and also serves to acquaint out-of-state people^^with the Montana 
projects, thereby facilitating out-of-state adoptions. * ' / 

Organizational Arrangements 

* » 
The consultant in charge of the project and two other -staff members 
,make up the unit responsible for the validation process in Montana. The 
/ number of^rojects considered for validation varies depending on the year 
and the funds available. On the average, three or four projects are 
validated annually. The validation schedule involves notifying schools 
throughout the year that applications for. validatioil are being accepted. 
The validation- process can take place at any point during the year. Over 
a two- to four-month period the following events take place. A 
validation request, together with project information and materials, is' 
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MONTANA (continued) 



submitted to the OPI* An in^^iouse Irevtew takes place to assess the 
likelihood that the project meets the state's critei^ia. The school 
receives, completes, and..submits ^ formal application form» The IVO team 
^Js selected, reads the project proposal, ^conducts an on-site reviewr^nd^^ 
announces -findings to th^ project perscuinel anci school administration* . 
The team then prepares its report and tpe chairperson sub*nits it to the 
OPI. ^If validated, projects may apply for demonstration status* 
Projects not receiving validation may request assistance from OPI staff* 



Support and Dissemijiation 

Projects whi^h'^ave been validated may apply for demonstration funds 
in accordance with the OPI philosophy that demonstration activity costs - 
shouVd not-'have to bffi assumed by the district; The state also pays for 
all of the necessary adoption/adaption activities with the district(s) . 
involved, including substitute pay, travel, needed materials and * ^ 
equipment, and follow-up activities* The OPI disseminates information 
about validated protiects via content ftrea-'cbnsultants ml- in proposal , 
writirfg^wofkshojSs, at which descriptions of projects are dis,tributed to ' 
participants. There ^s an intent to publish a brochure or mini-catalog 
this summer, which will provide additional information about validated 
projects. ' ^ ' . . \ 

Developers of validated projects, if they receive demonstration 
project monies, have different responsibilities -depending onj:he type of 
•project; for example, some are able to conduct large-group training and 
others work with smalT groups. The decision to submit the project for 
JDRP vafidation is at the discretion of ptoie^fzt personnel. - 



Contact Person * o - - ^ 
Judy Johnson 

Assistant Superintendent for 

Special Services ^ - 

Office of the Superintendent of Publ^ Instruction 
State Capitol Building 
Helena, Montana 59601 

(406) 449-3693 , ^ 



/ 
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' NEBRASKA 

Validat ion Process and Criteria Used 

' , 

Nebraska uses the IVD process arid the validation criteria, outlined in 
the Sharing Educational Success manual, \. 

Training PnocedurVs fo!* ValidatioVi " ^ • ' ' 

Training, patterned after the workshop organtfed by the federal. 
goyerRment, is provided regionally. ' " • . ' ^ 

Val idation Procedures ' 

Validation teams coiiiist of three people: one administrator, one ' 
evaluation' specialist, and one content area specialist. All validatJor^ 
,are from out-of-state, which adds- credibility tb^team judgments.' A two 
day site visit- is -an Integf-al part oj^ the team's^ork. >. / 

' ^ ' ' r ' 

. Organizational Arrangements . . ^ 

Six people at the SEA are responsible for-val idation. Approximately 
five programs are considered each year, of which one or > two .are 
ultimately validated. 

The schedule for Nebraska's validaticjn process is as follows: 

1. Schools are notified of the^process. 

2. ' An abstract is submitted by those. 

interested. \ 

3. Abstracts are screened. . February 1 

4. Those accepted are Invited to submit 

^ a. preliminary application. • ' - due February 15 

5. Preliminary applications are*" screened. . by March 1 

6. If a preliminary application is • 
accepted, an invitation to sjubmit an ' . ', 
application 'for validation is extended. M^rch 15 ' 



7. Applications for validation ia re screenied. \ April. 1' 
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NEBRASKA (continued) ^ 



8. If an^applicatiop for validation is 
accepted, the per'sons submitting it are 
invited to go through a training process 
and to hava an on-site visit. 

9. On-site visit is conducted. 



after April 1 



Programs not validated- can, request technical assistance, if they wish to 
re-appily fcfr validation. 



Support and Dissemination 

Validated projects ^ire 'eligible for funding for. dissemination activi 
ties. The D/D must agree to continjue the program, to acpept site visits 
from other educatorsi and to share information with others. 

JitlfrlV-C monies' are available to districts wishing to adg^t a vali 
dat^d program.. The state disseminates' inform^ion about validated pro- 
grams through press* releases, brochures, ^iit^r^wsletters and by providin( 
funding for awareness visits. 



Contact Person 

■Jack Baily _ - - 
Administrator of' Special Services* 
Nebraska Department of Education 
P.O. Box 94987 
301 Center Mall Street 
Lincoln, Nebraska 68509 

(402) 471-2481 
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NEVADA 



Nevadet does not have a fonnal validation process but does have a 
procedure for identifying educational projects and programs as> 
exemplary." In 1973, the Nevada State Department of Education produced a 
publlgation entitled "Evaluation Guidelines" to provide educational 
projects with information related to needs assessment, goals, objectives,, 
and program validation . The State Department of Education conducts 
periodic and routine reviews of educational project? and through this 
process identifies exemplary programs. Infbrmation about these ex^plary 
prograiris is disseminat.ed on atreferral basis and through periodic 
awareness workshops conducted by the SDE. - 



Contact Person 



*A1 Ramirez , 

Nevada State Department of 

Education 
^apltol Complex 
Carson City, Nevada 89710 

(702) 885-5700 



. * - * 
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- . NEW HAMPSHIRE • * 

* 

Validation Process and Cri teria Used 

New Hampshire uses a state-dtyeloped proces§ which U somewhat 
similar to the IVD and JDRP processes but which -is not as rigorous as 
those processes. The process utilizes a fivej»embnft" dissemination review 
panel focusing on effectiveness review critenf. Six criteria are used 
in the review of projects which apply for validation: (1) project 
objectives are clearly-specified; (2) implementation" steps are clearly 
described;* (3) target population is clearly described; (4) evaluati-on 
design is appropriate both- from a research peV'sp^ctive and to the ' 
objectives; (5) measures of ch^inge are valid and reliable for the 
specific population and the purpose of the project; and (6) results • • , " 
obtained are both statistically significant and .educationally meaningful.^ 

* * 

Training Procedure for Validators 

.. There is only orte /Internal validation teafin in New Hampshire. This 
team held one-day woPKShop to review criteria, generate consensus 
guidelines fpr operatioh of the panel, and examine examples of acceptable 
and unacceptable evidence of effectiveness. 'Since the team member*ihave 
remained the same, and projects are evaluated on a rolling basis, , 
improvement in the quaUty of the team.'s decisions comes from experience. - 

Validati-bn Procedures . 

Cu»*rently four staff members, within the New Hampshire Statjet 
Department of Education serve on the validation team. The validation 
panel must consist of at least Tive regular members, and pr6vision is 
made to incTude content specialist's when necessary. The current chair is 
the Director of the New Hampshire- OisseminatiorL Project, and the otber 
members are the .Director- of the Planning and Evaluation Unit, the- 
Cbnsultant fbr Exemplary and Developmental Programs in the Vocational 
Education Unit, the State Facilitator, and. the State Consultant for 
Language Arts and Reading. The Title IV-C consultant serve^^a v 
r\on-voting consultant to thp team. . 

No out-of-state validators are used by the state. The validation 
team works without a budget. , Because the team members are' State 
Department of . Education' staff , they are available to meet several times a 
year. • . ' • 

^ ■ V ^ 

Site vfs-its are made-when the te'am feels they are»nec'essary, and the 
team decides how many members will cpnduct the visit. 
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.NEW HAMPSHIRE! (continuedr ^ <^ 

f • 

Organizational Arrangements 
M ■ . " 
r One person in the State Department of Education, the Director of the-* 
New Hampshire' Dissemination Project, has primary, respons.ibi1ity for state 
'.validation activities. Five to ten projects apply for validation, each 
•ye'^Ht*. The passing rate is about 60 percent of the components seeking 
validation. (One project may have only one component; another may have 
three and only one of the«three may pass validation;«a third may have 
*five components and all may pass.) 

•Notices of validation opportunities-are distributed approximately 
four times a yiear. Wherya project submfts a request for validation, the 
•Director of the New Hampshire Disssemination Project visits the project 
director, and together they plan the strategy for subsequent actions.' • • 
Within three to four weeks/of the initial^ request, a validation team 
' meeting, is scheduled to^ review the project, anS.the project, is informed 
of the decision at the conclusion of the review meeting. \ 

* ~ ^ 

After the team decision is made, the Director of 'the Dissemination 
^ Project assists the project in disseminating the components^ that have ■ 
been validated and improving those that haveW. The project may -later 
request validation of previously unvalidated/components. 

. Support and Dissemination , . , 

Developers of validated projects can- apply for a diffusion grant 
through Title IV-C. Persons wishing to adopt, or. adapt a validated 
project can apply for a Title( IV-C A/A grant. ^ 

. The, state -disseminates information ^bout validated projects through 
RECON (New-England Regional Communication Network),, through. Project , 
•SPREAD, (the state prpmising practices file), and through the State 
Facilitator; xlirectors of validated projects are required to respond to ' 
requests' for information about the project. Many go beyond this minimum ' 
requirement and disseminate .materials, conduct workshops, invite 
visitations, and speak at conferences. Funded validated projects conduct 
* comprehensive dissemination efforts based on olbjectives negotiated with 
the TitJe^V-C office during the funding process. Also' Title IV-C4unds 
the New Hampshire Facilitator Center to provide technical assistance to 
all state validated projects. 



Co ntact Person * , " - . 

James V. Carr • *- >^ 

V Coordinator, ESEA Title IV 

Jew Hampshire State Department of Educatida • 
. 64'tlorth Main Street ' • 

Concord, New Hampshire 03301 - , 

' • (603) 271-3481 ' ' ' , * ^ 
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NEW JERSEY 

Validation Process and Criteria Used - , 

New Jersey uses the IVD process and also incorporates the folJowing 
criteria from the JDRP Ideabook ; change occurred, statistical ' 
significance, educational significance, generalizability, effects from 
the intervention, .and credibility/interpretability of the. evidence. 

Training Procedures for Validators 



New Jersey state staff conducted two, day-long reviews of the SES 
simulation; derived from tlie 1979 Denver, Colorado, USOE training 
session, to train six out-of-state and two in-house staff as validators 

Validation Procedures 

. New Jersey's certified validators consist of a school superintendent, 
an evaluation, specialist, and four teacher-training university 
'PCXtfessors, in addition to the state project director and a state 
valuatlo n-^ taff per^son. The-s+x-out-of-statfr-val-1<Jators read -vaHdat Ion 
applications once a year and. decide if projects are ready for a' site 
visit. A tvo-step procedural s used for practical and economic reasons. 
The- state attempts to ensiJfe that projects are "prepared for validation" 
before going through ^he expense and trial of a site visit. The use of 
out-of-state staff reduces the probability of political bias in the 
selection of projects. The one-day site visits are conducted by two 
validators.. One person usually handles exportability/cost questions 
while tHe other handles effectiveness/success matters. 

Organ i zat 1 onaJ-^angem^nts ^ ■ 

Basically two staff persons are involved in project validation— the 
Director .of Program Systems on a roughly 15 percent time basis and the 
Coordinator, of Evaluation on roughly half-time basis, providing IV-C 
project evaluation assistance. 

-Since 19^73, 56 programs have been validated. The number vat-idated 
per year has variied from 4 to 1?. On the average, some -25 projects are 
in development each year. Another 30 are in the demonstration, stage, and 
roughly 60 are under adoption. Roughly 70 to 80 percent of the programs 
developed are-validated. 
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NEW JERSEY (continued) 

The state's validat-lon schedule is as follows: * 

.May Orientation session, for eligible 

projects. 

.July-September Technical assistance and review^! * 

. September 22 Submission to panel. 

Panel meets to discuss sybmisslon. 

. Visitations by teams. 

Notification of decisions, regarding 
validation. 

The state provides technical evaluation assistance, through the 
Office of Evaluation, to all projects. The projects usually take 
advantage of this assistance at the beginning of their fourth year. 




Support and Dissemination ^ ' ^ ^ 

The state provides IV-C money to developers (0/Ds) for 
•dissemination— for the rrepro^luctlon of materials, brt)chures, etc., and 
for part of staff salarles'for dissemination activities. The itioney. 
prdvldea for dissemination in the fourth year cannot be more than 75 
percent of the project's third year development funding and ranges from 
$10,000 to $100,000, with the average being around $30,000.' In the^ifth 
year, the IV-C money awarded the jdemonstrator for dissemination cannot be 
more than 75 percent of the fourttigyear award. Overall, IV-C projects 
are limited to five years of funding: three years for development and 
two. years for dissemination. In 1980-81, New Jersey funded. 32 
demonstration projects (ESEA, Title IV-C). The 1980-81 figure includes 
one validated Follow-Through project and one validated State Compensatory 
Education R&O project. * ^ 

The state area provides IV-C Adopter/Adapter WA) grants to local 
education agiencies. The grants cover only in-district activities (staff 
development, etc.) and are usually for no more than $10,000 for one year, 
with the average being^ $7,000 to $8,000. N§w Jersey funded 64 ESEA IV-C 
adoptions in 1980-81. * «. 

DeveTopers must agree to participate actively in the dissemination of 
the project for at least a year. Most developers participate during the 
fourth and fifth years of the project. 
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NEW JERSEY (continued) 

_ State ijjformation about validated programs is disseminated via 
catalogs, brochures, awareness co^ferrences, and a computer search file 
system. * ^ 

' 4 

Contact Person 
Ronald E. Lesher 

Director^ Program Systems 

New Jersey Department of Educatloo 
Division of Operations, Research, 

and Evaluation 
P.O. Box 2019 

Trenton, New Jersey 08625 . 
(609) 292^7984 
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NEW MEXICO 



Validation Process and Criteria Used 

New Mexico 'uses a state-developed process for validating educational 
programs and practices. Central to this process is the Validation Jask 
Force, which reviews program nominations and self-evaluation forms and 
chooses members of the on-site verification teams. The major criterion 
for validation is that specific program objectives, as specified, must', 
have been met, 'Applicants must submit documentation of specif ic oltogram 
outcomes whfch indicates the extent to which objectives have beertmet. * 

Accep table pr og r am outco m es ca n be speciric to stude n ts,. Leachers; 

.administration/management, parents/community-, or other areas related to 
school improvement. Both statistical and narrative/subjective data can 
be submitted. - 

Training Procedure for Validators 

There is- no formalized training procedure for verification team 
members. Members of a verification team meet before the on-site visit tn 
J^exlew ail availflblejnateriaJs. aniJ:o_ cQocdinate_aa^ vi-sit 



itself. 

t 

t \ ' 

Validation ^fecgaurgS" ^' • . 

The Vallidat ion Task Force consists of four people, including a. 
representative from each of four State Department of Education units: 
Evaluation, Elementary and Secondary, Vocational Education, and ESEA 
Title IV. On-site verification teams include three people,, usually a 
state Department of Education person, a local education agency person, 
and a university faculty member. All members' of ^he ver'if icatiorv team 
a»*e from within the state because they tend to be familiar with existing 
state programs, they understand and are invested in the concept of 
within-state school .improvement efforts, and t|1e State Department of 
Education prefers to use volunteers rather than paid consultants/ 

One-day site visits by the' three-member venif ication teams are part 
of the state's validation procedure. \ 

Oraganizational Arrangements ; 

Two professional staff members in the State Departinent of Education 
are responsible for validation activities.' During the 1979-80" school 
ye«r, in a pilot test of thg state validation plan, eight Title IV 
projects were invited to apply«for validation. All of these— 6 Title " 
IV-C projects and 2 Title IV-B projects— were validated. Beginning in 
the 1980-81 school year, the process was opened to j(ll interested • 
programs in the state. 
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,NEW MEXICO (continued) 



♦ 

Informational presentations on the state's validation procedures are 
made at the state administrators' conference and at the state«-Title IV 
conference in the Fall. For 1980-81 nomination forms were due a^ the 
State Department of Education on September 30 and self-evaluation forms 
'were due on November 14. On-site verification visits, possibly ia 
conjunction with regularly scheduled "in-depth visits" by State 
Department of Education currlcular specialists, were conducted in Spring 
lS84-^d-not4f4€ation of- results was made in late Spring. 

Projects which are not approved for validation can request technical 
assistance frort the State Department of Education. It may be possible' to 
provide validation-related assistance to interested districts by mfenbers 
of "in-depth visit" teams from the various State Department of Education 
curricular units during their regularly scheduled district visits. 

X 

Support and Dissemination 

• For the 1980-81 school year, two validated Title IV-C projects have 
been awarded, funding' for dissemination/demonstration purposes. Each 

Foje€t-re€e4ved-^2v500. . «fla^unds-are'-current-4y-ava44ab1e--f or school 
districts which are interested in adopting'' adapting a project. 

The state disseminates informatidn about its 'vjalidation process and, 
validated programs in several ways: the state maintains a collection of 
one-page descriptions of validated programs in the Department of 
Education's Resource Center which are available to interested schop.l 
districts; state staff make presentations regarding the state validation 
process and validated programs at statewide conferences and workshops; 
and the state mails out brochures describing the validation process and* 
the current program col lection .to district superintendents. 

* Validated programs must be willing to operate the program for an 
additional two years; "respond to telephone and letter requests for 
program information and keep a record of those requests; host visits from 
interested persons; provide printed information about program management 
and operations to interested districts; and lend available program 
materials. 



Contact Person 

= y . - 

Susan Brown 

Coordinator, ESEA Title'IV ) 
New Mexico State Department of Education 
Education Building, Room 102 
Santa Fe, New Mexico 87503 

(505) 827-5441 
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NEW YORK • " " 



Validation Process- arid Criteria Used 
' : '■ 

New Vork has developed its own procejiijre for validation of , Title IV-C 
funded projects. Applications can be filed at any time, and a 
preliminary review begins immediately. Site visits are conducted if 
warranted. A site visit team report to the State Education Department is 
/the basis for the decision about validation. Projects which' seek 
(validation are required to submit a stat«nent regarding their objectives, 
which must be in fneasurable t erms; ass essment techniques; the evalua£ion 
design; a summary 'of the data; a statfstical analysis if one was 
performed; arid -a statement about the educational significance of the- 
project or program. .Additional requirements are a statement aboutvthe 
context within which the project operates, a description of the prdject, 
and cost dat?. " - - ' 



Training Procedures for Val idators 

New members -are nominated by current members' of the Validation Team, 
district superintendents-, or State Education Department personnel . 
Members are picked on the basis , of their .backgrounds in evaluation, 
administration, or*prograro content^xpertise. Train.ing sessions, are 
conducted by the Coordinator of Validation and feature qase studies of 
validation evjdence. - - 



Validation Procedures 

Projects are prescreened by a state-staffed preliminary review team. 
The team consists of an evaluation specialist., a content specialist, and 
a program manager in Title IV-C. The site, ws'it team includes an 
evaluation speci.alist, a conteqt. specialist, and a school administrator. 
Although teams are.usyally made up of In-state validators, out-of-state 
validators may be used. if they are located nearby. The practice of using 
in-state validators is preferred -because it is' more cost-effective. 
On-site visits are 1 and 1/2 to ,2 days in length and are conducted, at a 
minimum, by three people.- Occasionally, a State Education Department 
representative wiTl participate in the stte visit. 

• ^ 

\ 

, Organizational Arrangements 

There ar^ six staff, members in the division responsible for 
validation;' one of these people supervises the process. Fifteen to 
twenty iridividuals' in other program units have been involved in 
preliminary y-evlews of applications. Approximately 25 projects 
voluntarily apply for validation each year. From December 1976 through 
December 1980, }0Z applications were reviewed. Of those completed, 50 
were validated and 52 were rejected. 
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' ^ NEW YORK (continued) ^15. 

I ' ' 

There is no specific time schedule for submission of applications. 
Preliminary review is completed within two to five weeks after an 
application is received. If a site visit is necessary, a team is 
selected and the visit is scheduled four weeks in advance. The team- 
report is submitted to the State Education De|iartment one to three weeks 
after the site visit.. 

If a program or project is not validated, the developers may be given 
strong encouragement to reapply. Reviews of applications contain 
diagnostic commentary about the nature of the evidence and the way the 
evidence is presented tn the^ application. % 



Support and Dissemination 

After a project or program is validated, the state provides funding 
for continued development. The state also provides LEAs with money <for 
demonstration activities. Districts which would like to adopt or adapt a 
project or program can'S^jbmit applications for funding to the state. The 
state does not prohibit school districts from replicating programsrwith 
other sources of money. 

* . - . 

• The state disseminates information about validiaited projects and 
programs through publication of a catalog, •iNew. York State Education 
Programs That Work," and through the NDN regional facilitators in New . 
York. Projects or programs that are validated must agree to sarve as 
demonstration projects if the state has determined a widespread need for 
adopting,.^ project. 



Contact Person / * 

Richard L. ^gelston . 
Validation Coordinator/Title IV-C 

Program Mana^pr 
Room 860 EBA , ' 

New York State Education Department 
Albany, New York 12234 

(518) 474-2380 
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NORTH 'CAROLINA 



Validation Frocess anci Criteria Used 

iria are used. 



, IVD and JDRP procedures and criteria are used. 



Training Procedures for Validators 

IVO validators were trained by ROEP IV. A sirpilar activity to train 
< regional validators was conducted in-state. ^ " * 

Validation Procedures/ * 

Validation- teams include specialists in evaluation and-content/ 
exportability. Depending upon the nature of the program being validated, 
teams are composed of in-state and/or out-of-state educators, This-mix 
tends to enhance the team's credibility. On-site visits of two or three 
days are routinely conducted by the three-person team. 

^ ■ . . _ ^ _ . \ ' . 

Organizational Arrangements 

One SEA staff person is responsible for coordination of validation. 
He assists* oth^ SEA staff members who are working with programs which 
might be validated. While no established number of programs is 
considered for validation -each year, there are usually several programs- 
which request validation. To date, all programs which have been 
dominated have beep validated. ^ 

The Division of Development of the SEA discusses the validation " 
concept with each new development project at the beginning of the funding 
year (July). When a project is nominated (usually in its second year), a *- 
timeline is established by the project and the validation coordinator. 
By April 1, all applications to serve as a demonstration site must be 
submitted by programs likely to-Ae validated. Similarly, ppst-validation . 
procedures are planned on an individual project basis. Plans include the 
provision of technical assi'stance ta projects which are*not validated if, - 
in the judgment of the validators, they might be-validated in the future.' 

Support and Dissemination ■ 

Depending on availability of funds, developers of validated programs 
are funded as demonstration sites. Adopter grants are also available for " 
training, travel, and materials,. The local education agency is also 
.encouraged to provide funding to eldopt validated projects. 
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- NORTH CAROLINA (continued) ) 

DeVeloper /demonstration projects agre^ to provide awareness, 
training, ejnd technical assistance to potential and^actual adopters. The 
eight regional facilitators (SEA staff) disseminate information about 
validated programs* Also, because SEA content specialists are involved 
in this program development, they disseminate validat-ed program 
information to- North Carolfna school districts* 

Contact Person . . ^ 




Mr. Henry Helms 

Director, Division of Development ' - * 

North Carolina Department of Public ' 

Instruction / 
Education Annex I *' ' 
^'kaleigh. North Carolina 27611 ' . ^ 




1^19) 733-7018 ^ , ' . " 


• 

• - 


1 • . ' \ - : 




1 
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f 


% 
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NORTH- DAKOTA i 
f ' ' 1 ■ , 

Validation Process\ind Criteria Used , 

North Dakota uses the IVD process and the validation criteria ' 
outlined in the Sharing Educational Success manual. 

Training Procedures' for Validators . 

Validators participated in regional training sessions. Local 

training by trained- validators is alsio provided. 

Validation Procedures. _ • 

Validation teams are composed of three persons: an evaludfcrf 
specialist, a content area specialist, and a fiscal/administr,ative 
■specialist. The three team members- are all out-of-state educators," since 
it is felt that they will be less b'lased, for or agai'nst, i given program 



-During the on-site visit, which lasts a minimum of two days and may- 
run /no more than three, the team is supplemented by state education' 
agency directqrs. . 4 

. , J . ^ ^ ... 

OrganizaU^nal Arrangements^ * ^ 

' One person at the SEA has respQrisiWllty for validation. While nine, 
programs Were originally Validated, and three programs are annually ^ 
validated, there is no fitxied schedule for validation. Rather, the 
procedure i? implementedM>ased on demand and on initial qulaility review by 
the state validation coordinator, , 

Programs which fail to be validated are provided technical 
assistance, again, on a demand basis, 

• V * , 

Support arid Dissemination 

Developer/demonstrators (D/Ds^P HpHln technical assistance on 
request to the SEA. Adoptions arej^^JP^-accordanplTwith^^Laite 

in 




objectives ~sa that grant funds fdi^D1|j|BoHifiJ|5*^can have Irppact in terms 
of state needs. At present validated' programs *are no.t required to seek 
state revalidation. " 



. .146 . 



NORTH DAKOTA (continued) 



For the most part, the D/D's primary ."responsibility is to create 
awarertess and to host visitors to the project. The State disseminates 
information about validated programs through state publications targeted 
to teachers and to administrators. In addition, the state works with 
projects to prepare JDRP submissions. « 



Contact PerjsojL 



Dr. David Lee ^ 
Director, ESEA IV-C 
Department of. Public Instruction 
State Capitol 

Bismarck, North Diko1# 58505 



(7(y^) 224-2391 
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OHIO 



Validation Process and Criteria Used 

^- , ■ 

Ohio uses a state-developed validation process and publishes a manual 
for Project Validation in Ohio to structure the process. Ohio's 
validation criteria include: * • " 

1. Evidence of success by pr»oject objectives. Emphasis is on 
statistical proof of the treatment model. , ^ 

1 ^~~T. Start-up, management and operational costs niust be 

^ speci/ied for producer and potential adopter districts. 

3. The project must continue to operate past/£he federal 
. funding period. 

4. Appropriate target audiences must be specified as well as 
the heed for this project in other districts. 

» 

5. The project must identify key learner activities, 

- inaterials, and processes necessary for an adopter to obtain 
^ I similar results. 

6. Key instructional variables must also be specified. 

7. Cr^iticalr project activities and elements for success must 
be- specified. 

8. If appropriate, the pros and cons of partial adoption must 
be discussed. 

9. Personnel- needs must be specified. 

10. Any special problems for potential adopters should be 
discussed. ^ ^ 

11., A description of the evaluation design is required.. 

12. A description of the activities and procedures used in 
attaining the objectives is required. 

bl Developed processes or products must be exportable to other 
X districts. 



Training Procedures for Validators 

'The Ohio Department of Education, Division Tof Planning and 
Evaluation, is responsible for the conceptualization, and implementation 
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OHIO (continued) 

of program validation. Division siaff nominate, select, and train a core 
group of people regionally dispersed in the state for the validation . 
teams. '.New validators are oriented by Division staff and team leaders, 
using the state's Manlial for PT*oject Validation , via phone and mail. An 
on-site pre-visitation orientation meeting is held for project developers, 
and Validation team members the evening prior to the two-day on-site. 

Validation Procedures 



ERIC 



'rue State validation procedure consists of two-day site visits, 
usually in March, by three person validation teams consisting of an 
evaluator to assess evidence for success, a .curriculum persoajp. assess 
exportabi.lity, and a financial/administrative person to assess economic 
efficiency. Only in-state validators are used for reasons of cost and 
quality control regarding the 5tate process. Sites are chosen through 
review of a self-nomination form completed by interested projects. « 

Organizational Arrangements ^ 

. > Two Division of Planning arid Evaluation staff manage the state's 
validation process. The number of programs considered for validation has 
varjed from 5 to 15 per year. On the average two to three projects have 

1)e€n validated per year. 

The state schedule for validation is.listed below: 

April -December District s61f-nomi nation f or ,the 

validation review process made^Sy the' 
superintendent of the developer district 

December- January State screening of the 25-page 

self*-nominat1on form for completeness 

January-February ' ^Organization of the on-site teams begins 

March ^ On-site project reviews are conducted 

ApriT Awards regarding validated programs are 

announced. 

Programs that are r^ot validated may re-enter the process. 

Support and Dfssemi nation • ' 

Since 1974, the Division of Planning and Evaluation has conducted an 
Adoption Grants Program. Over 400 grants were awarded in the first three 
years of;the program. The Division also^uppor^s D/D districts In that 
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validation grants are awarded to distrjcts to .cover al\ the necessary . 
costs of the orientation/tr.aining me^ifig and the on-site validation 
activities of the reyiew team. .Hina'%; Division staff support the 
district through the preparation, of a Coordinator's ManuaJ if the project 
is accepted in the Adoption Grants Program, • - " 

The extent of developer participation in the dissemination of the 
project is left to their self-determination. Division'staff frequently - 
encourage projects to seek JDRP certfif ication.. 


■ c 




,-\ . •' . 
The state disseminates information about validated projects through ' 
the "Forword" newsletter, at the.annuaT spring awards conference'^and ' . 
«■ through the Adoption Grants Program if accepted. The State Advisory 
Council for IV-C considers validated projects for Adoption Grant 
inclusion. * 






Contact Person - ... 


* 




George, Harris ' > , * ' 
Educational Ctinsultant " • - ~ > 
J)1visidn of Planping and Evaluation .' ' 
Ohio .qiepartment of Education ' • ' 
Room 802 ' * 
Ohio Departments Building ' i 
65 South Front Street '< 
Co^lumbus, Ohio 43215 




1 


(614) 466-4838 . " < 

\ 

* 

♦ 

m a 
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OKLAHOMA , ^ 

Validation Process and Criteria Used 

Oklahoma uses the IVD process, and the validation cri-teria outlined in 
the Sharing Educational Success manual. 

Training* Procedures for Validators 

To date, validation team members have received training in seminars 
,oi\^ the IVD process. They are certified IVD on-site validators. 

Validation Procedures < 

Oklahoma uses in-state teams composed of two to three people, 
including both evaluation specialists and content specialists. The use 
of in-^state validators reduces expenses for the state, and the 
validators are more readily available. As part ^of the validation 
procedure, the team spends One day visiting. the project- site. ^ ' 

* 

Organizational Arrangements 

ThB Title IV-C staff Is very small* Other departments w4thin Title 
IV assist'Title IV-C 'staff in validation, activities. Two to ten 
applications *f.or validation are received each year, and four to five / 
projects are validated. ^ • '^^ / \y 

\ I ^ ^ X ' 

' The sequence of events in the validatidj^ process is as follows: 
Announcements are,maide to schools 

Interested IV-C projects should notify State Department of Education 
Schools that receive favorable response will. submit one page on ^ * 

accomplishments ^ ^ ^ ^ 

< ' Identified sucejS'Sful projects are notified 
Validation team is appointed 
Site visit date is determined 
Final submission of application 
Validation team reads the report* 
Validation team makes the ^on-site visit"* 

Validation team makes their report to State Department of Education- 
School is notified o-^ •approval or disapproval. 



OKLAHOMA (continued) 

_r.itle IV-C_ projects which are not validated but wttich show potigtitial 
can request technical assistance from the State Department of Education* 

Support and Dissemination . • j 

The- state supports the activities of developers of validated programs 
and projects through the awarding of^ competitive Titlfe IV-C grants. 
School. districts which are interested in adopting or adapting a project 
can also apply for funds on a competitive basis. 

The state prints a directory of all Title IV-G projects operating for 
th€f current year. This directory is mailed to every school district in 
Oklahoma. Special articles on state developer/demonstrators are included 
in the "Oklahqma Educator," a-regular publication of the State Department, 
of Education that goes to all school districts in Oklahoma. v 

Developer/demonstrators are required to respond to requests for 
information from other schools and to receive visitors from schools . 
interested in adopting the. project. If a school officlalljf adopts the 
project, then the developer/demonstrator is available to give technical 
assistance to the adopting school. 

»- 

Conjtact Person 

Al Underwood ' . 

AdministratoTv-Title IV-C 
Oklahoma- State Department 

of- Education 
Oliver Ho^ge Memorial Education 

Building . ' • 

2500 North Lincoln Boulevard 

Oklahoma City, Oklahoma 73105 _ . 

\ 

(405) 521-3694 
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OREGON 



- • Validation Process and Criteria -Used 

Oregon makes use of a state-developed validation process In which 
projects under consideration are given ratings in six areas and a final, 
overall rating. Materials are reviewed for the evidence they give that 
the program is complete, transportable, and self-sustaining. The 
utilization of project materials at 'the test site(s) recetVes a rating, 
as does the likelihood, that a "typical district" could successfully adopt 
^ the program. The process requires that the project be successfully used 
by an adequate number\of teachers, and a rating is also assigned based on 
•evidence that the "^typical teacher" can use or has used the .program - 
, successfully. Likewise an adequate number of students must have 
experienced success with the program and because of the program. 

- ■ • ' • • . .-. • 

Traintna Procedure for Validators 

The Oregon validation process. has. nd special training procedure. The 
process is, however, characterized by a long-term association between . 
project staff* and the validation team, whose activities follow from the 
state's validation guidelines and rating form. 

Validation Procedures " . \ 

One Title IV-C project. officer, three ^tate Advisory Council members, 
^and an Oregon Department of Education (ODE) subject matter specialist 
» generally comprise the. monitoring team for the project under 
-consideration. / Monitoring visits generally take place twice a year 
throughout the development pha§e of the project. Recommendations for 
validation are determined by the cumu-latlve findings made. during these 
visitations over a period of several ye.ars. The final decision Is m^de 
by the full Council based on a review of the monitoring team' report'* 

Organizational Arrangements ^ 

The ODE Title IV-C unit has' a staff of two people. Two or three 
- projects are considered and validated each year. 

The final year of develo^xnent is the project's field test year. When 
projects are notified of their funding, they are encouraged to collect by 
early spring the project data to be reviewed by the validation team. 
' This timing makes validated projects available for adoption the following 
year* The project may also receive a dissemination grant. 
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OREGON (continued) 



Support and Dissemination 



Title IV*C provides between $25,000 and $100,000 for dissemination 
purposes to validated projects. The grants are issued on a conpetitfve 
basis and ^re for one year. ' Substitute pay, travel allowances, and some 
materials and staff training costs are provided. Technical assistance is 
readily available from Title IV-C staff and other validation team members. 



Dissemination of information about valid^ited projects occurs in 
several ways. Some projects have grants to disseminate information about 
their projects. » Disscfmi nation activities include appearances and 
presentations at conferences, educational fairs, school board ^etings, 
and other appropriate meetings in response to invitations. Validated 
projects also provide technical assistance to A/As, respond to requests 
for materials and information, conduct training, monitor adoption 
procedures and assist A/As in their program evaluation. 

The ODE supports the disjseminatiqn effort by publishing descriptive 
-information by means of cata'^logs listing programs available for districts 
to adopt; through newsletters developed by ODE programs; in articles • 
published in the ODE newspaper, EDU-6RAM, which is published six to seven 
times^a year; and through the information services offered by the ODE 
^ Resource Center. 



Chuck Nelson, Program Specialist 
ESEA Title JV-C 

Oregon Department of Education 
700 Pringle Parkway, S.E. 
) Salem, Oregon 97310. 

(503) 378-6799 . 



Contact Person 
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PENNSYLVANIA 



VaUdation Process and Criteria Used 

A variation of the JDRP panel procedure is used in Pennsylvania. 
Districts applying to validate a project must submit a ten-page 
validation application paper describing the project's need, objectives^ 
program activities, evaluation outcomes, materials, and exportability to 
a 30 member panel, the Pennsylvania Diffusion Panel, which is convened 
annually. A mix of IVD arid JDRP .criteria is usjfid in evaluatiag 
submissions for validation. Panel members rate each program, using a 
four-pag6 criteria checklist, on its evidence of program need, 
effectiveness, capability to diffuse, adoptability, and cost to diffuse. 

{ ' ^ . ' ' 

Training Procedures for Validators 

New members of the Pennsylvania Diffusion Panel receive a one to two 
hour orientation from the Panel jCoordinator regarding the state's 
validation rating procedure and criteria.- , 



Validation Procedures x \ ' . 

As was noted above, the 30.membl^r Pejagj^yWa Diffusion Panel is 
convened annualljr to review submissions for validation. Members of the 
Panel are selected to represent a cross-section of specialties- 
evaluation, content areas, research, and administration. Only in-^ate 
people are used and of these roughly two-thirds are Pennsylvanie^ / 
Department of Education staff. The state does no.t ordinarily use site 
visits in the validation procedure and chose the Panel process for cost 
and convenience' reasons.. Each of the project papers submitted for 
validation is read by approximately 15 Panel members and must be approved 
by two-thirds of the readers to become validated. 



Organizational Arrangements ^ , > 

Pennsylvania has two Title IV-C staff who are primarily involved in 
the validation process. -On^ sljaff member monitors the D/Ds and 
coordinates -the Pennsylvanta Diffusion Plan. The* other allocates IV-C 
monies for the plan. Approximately 25 programs are considered for 
validation each year and of these 12-15 are val ideated. 

f 



ERIC 



145 



• 155 



PENNSYLVANIA (continued) 



Since 1976 a total of 84 programs have been validated. The state's 
schedule Yor validation is as follows: 



August 1 j Announcement of requests for* validation 

and panel meeting date 

October 1 ^ Deadline for submissions of. applications 

for validation 

October 15 Pennsylvania Diffusion Panel convened to 

review applications 

February ^15 Orientation for directors of validated 

projects regarding D/D 
responsibilities/procedures 
• * . 

March/April Open house week by D/D projects to 

facilitate awareness 

0 

* i 

Programs^hat are not validated are asked to make recommended changes and 
resubmir^he following yeaY. The Director of the Panel suggests 
technical assistance sources on a request basis. 

* *' 

Support ahd Dissemination , » ^ 

• The ?tate does not provide direct support for dev(eloper/demonstrators 
(D/Ds). xThe state, however, does provide grants to tne adopter /adapters 
(A/As) of^^ate .validated projects— approximately $6,000 for the ini^fal 
year. These monies can be used to contract for support (travel, 
workshopfS^ands materials) for D/Ds. Developers (D/Ds) must agree to 
participate in ^he diffusion of their projects, if validated. D/Ds must 
conduct awareness or training workshops, prepanne program materials and 
generally cooperate with A/Asv The PDE/RISE Catalog of IV-C Projects, 
brochures, newsletters, and education association jodrnats are used to 
disseminate information on the state validated projects. 



Contact Person 
1 

Waldo G. Weaver, Coordinator 
Pennsylvania Diffusion Network • 
Pennsylvania Department of Education 
Box 911, 333 Market Street 
Harrisburg, Pennsylvania 18108 

1 717)^ 787^7372 " 
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^ RHODE ISLAND 



Validation Process and Criteria Used 

Rhode Island uses- the IVD process and the validation criteria 
outlined in the' Sharing Educational Success manual. 



Trjilning Procedures for Validators 

Validators attend national^ training workshops. conducted by the 
. Department of Education. 

Validation Procedures , * * 

It: f ft 

Members of the validation teams used by Rhode Island are specialists : 
in one or more of the following areas: evaluation, finance, 
dissemination, and content., Under the old IVD system, out-of-state 
validators were used to review projects. Because of financial 
considerations and the pew flexibility of the IVD process, -in-state 
validators are now used. Site visits are part of the validation 
procedure. ; They last 2 and .1/^2 days. and are conducted by three or, four , 
people. . . ' 



Organizational Arrangements 

Two professional staff members in the State Department of tduqation 
are responsible for coordinating validation activities. In addition, all 
Title IV staff contribute some portion of th^ir time a^ needed. Two to ' 
three programs or practices are submitted for consideration each year. 
Of these, on the average, one or two are validated. ' ^ 1 

An announcement goes out to school -d is tr'^cts in January of^each 
year. Letters of intent to apply must be received by February 28, and 
the fulKapplicatioYi must be submitted, in March. Site v'isits are 
conducted in April, May, and June, and awards are made by June 30. In 
past years, projects which were not validated couTd ipply for an 
evaluation grant to assist them- in establishing proof of 
effectiveness/success. Such grartts" are no longer available. However, 
projects can request technical assistance from th? State Department of 
Education. Technical Assistance is also available for. projects which 
intend to prepare. a submission "for JDRP approval. 




. RHODE ISLAND (continued) . 



Support and DissewinatiQn 

_ r 



The state supports thf" activities of developers" of validated projects 
and programs by paying the cost of demonstrating and disseminatijig the. 
project. School districts which want to adopt a project cjan apply for a 
grant from the state. ' ^ 

Rhode Island disseminates information about its validated projects ' 
and programs throu<|h RECON, the New England Regional Communication 
Network. Oeve Topers of validated projects are responsible for 
disseminating information- about their project, providing inservice' 
programs for adopters of the project, and helping adopters develop 
applications for adoption grants. 



Contact person ^ 

DHfJohn Custer . 
Consultant, Title IV 
State Department of Education 
'2'34 Promenade Street • , 
Providence, Rhode Island ' 02908 






SOUTH CAROLINA 



Valiclation Process and Criteria Used 

The IVD process is used for validating programs and procedures where 
developers plan to apply for JDRP approval. For other programs, the • 
criteria and procedures of IVD are' used except that the composition of 
the validator team is different: 



Training Procedures for Validators 

Validators are trained by South' Carolina Department of Education 
staff (certified validator trainers). 



Validation Procedures 

The validation team is composed of two evaluation specialists and one 
content specialist. If necessary, because of the program's scope, the 
team can be expanded. The team chairperson^ is from out-of-state but, for 
economic reasons, other team members are from within the state, the team 
visits the program site for one or two days. 



Orgarrizational Arrangements * ' 

One professional at the SEA is responsible for validation. She is 
assisted by other SEA staff io the program^off ices as necessary. The 
number of program* considered for validation has been two per year, 
although it is anticipated that their number will double in 1981. On an 
average, one program per year is validated. * 

No fixed schedule for validation exists. As the need arises, 
programs are scheduled for team visits. A program which is not validated 
is examined for potential validation. If such potential exists, 
technical assistance is provided by the SEA evaluatibn consultant "or SEA 
program consultants. 



Support and Dissemination v 

ESEA IV-C funding is used for support of both D/Ds and projects which 
are interested in adopting or adapting (A/A). D/Ds are responsible for 
statewide dissemination and training A/A staff Dissemination is built 
into IV-C project proposals. .•• ^_ 
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. south' CAROLINA 

/ 



SOUTH' CARQLINA (continued) ■ / 

• . ■ ' • ■ . • / 

Contact Person , • / 



Mr. Joseph ,C. Pate 

Chief Supervisor, Title I¥-C Section 

Office of Federal Programs 

South Carolina State Department of Education 

1429 Senate Street 

Columbia, South Carolina 29201 

»- 

(803) 758-2911 
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•^^UTH DAKOTA 



Validation Proce ss and Criteria Used 

' South Dakota^ dses the IVD process* and the validation criteria 
outlined in tfie Sharing EducationaV Success manual. 

■ • • ,/ - ■■ , 

Training Procedures for Validators t ^ 

The validation team leader was trained at a U.S. Department of 
Education training conference and has subsequently trained bther in-state 
people. 



\tol1dat1on Procedures 

'' '■ ^ \ 

The teams are coftiposed of evaluation and curriculujn specialists from 

universities. Out-of-state team menljers were used in the past, but for 

econofhic reasons and for profess ionapgrowth, two team members will be 

in-state jth.is year. 

' « 

Applicants are presCreened. Three people are on the site visit 
team.. The visit lasts between 1 and 1/2 and 2 days. 

Organizational Arrangements ' 

One person at the SEA is responsible for validation. Each year, one 
or two programs apply for validation, and the applicants have been 
successful. 

In October, meetings are held with potential projects. Fn December,- 
the projects are nominated for validation. The on-site visits occur nn 
April, and in May decisions to validate are made. 

Support and Dissemination 

^ The state funds projects which have bfeen . validated for continuation 
and for dissemination. During this perfod, validated projects are ' 
encouraged to prepare for JDRP submission. 

« Adoptions are supported with Title IV-C funds. Additionally, the 
state provides grant writing woirjcshops to encourage ,wider adaption of 
validated programs. 

\ ♦ r 




C - 151 



SOUTH DAKOTA "(continued) 



The -state also helps disseminate information through puyications to 
administrators and to teachers. 

^'"^ • ^ - . ' 

Contact Person 
Gene Dickson 

Director, Title aV-C and State Facilitator 
Department of Elementary and 

Secondary Education 
Xneip Building 
Pierre, South Dakota 57501 

(605) 773-3395 . , ^ 

• v .- ' . ■ 
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TENNESSEE" 

Valid ation Process and Criteria Used • 

A staterdeveloped variation of the IVD process is used for< 
validatiop^ The major criteria for program validation include: 

^ Effectiveness 

Clarity of objectives 
Availability of data 
Exportability 
— ^ Training packages 

* Materials lists as packages 



Training Procedures for Validators 



Valrdators are trained during a one-day training sjninar. 
Validation Procedures 

. *v 

Generally, the validation team is composed of Sfte content specialist, 
one evaluation special ist,%hd, one exportability specialist. Tnese 
people are all in-state educators because there isno money budgeted to 
support out-of-state validators jand because of specific requirements in 
the state validation process. 



r The tfeam of three or four' people makes an on-site visit of two days 
as p^art of the jceview procedure. ^ 



Organi2atiope(l Arr^tn^Bneots V 

Tour SEA sWf are direct4x->^esponsible for coordination of the state 
validation proceoure, although fifty SEA staff persons may be available 
as. part of the review team. ^ l.,,.,,,^**^^ 

E4ch year thVee to five programs are considered for validation, of 
which two to four annually are validated. 

Although a fixed schedihe has not been established, the following ^ 
series of steps are undertaken within the validation process: 

1. Preliminary conference / 

2. Letter to lEA inviting- application for validation 
'3. ' Response from LEA ' ^ 

4. First submission from LEA 
^S. Revisions, as required l 
^ '6. Identification ^f review^team 
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TENNESSEE (continued) 



• 7. On-Site visit' 

8. Recownendation by Title IV-C Advisory Council, based 
' • on team recommendation 

9. Notification by Commissioner of Education 
10. SEA publication of awareness materials 

Programs which submit but'are not validated e»re invited to seek 
technical assistance from Developmental Programs office staff, ' 



Support and Dissemination 



Programs which are validated may apply for demonstration funding for 
two years. ^ This funding comes from ESEA Title IV-C. In any event, 
successful developers must continue the program for a minimum cyf two 
years* Similarly, LEA adopters can be supported ^y funds of ESEA Title 
IV-C. . ■ 

i' ' - 

The SEA disseminates information about validated programs through 
awareness conferences\ brochures, and mailings ta school units. 



Contact Person 

Barbara Oakley Adkisson 
Developmental Programs 
Tennessee Oep^Crtment of Education- 
lie Cordell Hull' Building 
Nashville, Tennessee 37219 

(615) 741-1996 ^ 
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TEXAS 



Validation Process and Criteria Used 

A* st.ate-developed process is used for* validation. In-order to Be „ 
"eligible for validation a program must:' 

. • , 1. Havfr been in operation*two or more years 

2. .Continue in operation for -two additional years 

3,. Be observable .in operation 

4. Have been designed to meet a documented 'need 

5. Provide formal written objectives, the evaluation design, 
and documentation showing evidence of success in^ meeting the 
needs for which it was designed 

6. Ben^eplicable in a similar school district or regional 
education service eienter ; 

• ^' 7. Be irt conpliance. with' applicable regulations if financed 

partially or totally by categorical state or federal funds * 

■" . 8. Be. open to a verification visit by an individual or 'team 
• ' designated by the state' validation steering c6mmittee\ 

Additional criteria for effectiveness are developed to be' appropriate 
to the particular program being examined* 

Training Procedures for Validators r • * 

Specific training for all team rflembers has-not'^TJwn possible because 
of financial constraints. JCWever, fn 1979-^0, a half-day training, 
session for^^bout 20 people was conducted^ Training focused on purpose 
of validation, criteria, procedures, and instrumentaton. 

■ • y ' 

Validation Procedures 

/ -l 

A val1diit4en team of two to five peopl6, representing content, 
evaluation, and administrative specialists visits the'site for at least 
one day. ' ' ; , ' 
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TEXAS (continued) 



Organizational Arrangements 

^ Two professionals at the SEA are actively involved in the validation 
effort* Each year, about 100 programs are considered for validation, of 
Which about half are ultimately validated* The sequence of events is as 
follows: ' ^ ' 



May 1-31 



June 1-October 31 



NQv^m()er 1-30 



Announcement to school 
superintendents, regional 
educational service centers, other 
special mailing lists, SEA ^ 
specialists 

Nominated program directors 
complete detailed self -report form 

SEA specialist and task forces 
(including outw)f -agency people) 
review self -reports, from program 
directors and recommend visiting 
»or not 



December 1-May 1 
mid-May 



Visit programs 

Statewide steering' committee meets 
to review reports of programs" 
visited and to make selections 



September 1 



Annual directory published 



Those programs Which are not validated are provided technical 
assistance on an informal basis. ■ Moves are currently bejng made to 
strengthen this procedure* 



Support and Dissemination 

Those programs which are validated and which become demonstration 
sites undertake nine obligations: * , 

1. Serve a term of approximately two years, ceasing when*the 
nornjal J program schedule ends during the suniner following the 
second year of^the term 

2* Receive reasonable numbers of visitors by appointment at 
de^ignat^d times appropriate to program and staff 
scheduling, stuc|en^ needs ^ etc* ' 
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TEXAS (continued) * ' . 

3. Provide a place, for orientation df visitors-: 

4. Hold brief visitor orientation and debriefing sessions 

5. Appoint a staff member to he responsible for orienting.and 
guiding visitors , • 

6. 'Provide visitors wltri a brief printed program description 

and schedule 

7. Have visitors sign register provided by the Texas Education 
\ Agency and send completed register sheets to TEA 

♦ 

1^ 8* If invited, send a- representative to present the program at 

a regional or state awareness conference . ' " 

9. Provide 'written information which would be useful to another 
. agency considering adoption or adaption. , ^ 

^ * • * 
♦ The-state disseminates information through quarterly newsletters to. 
local superintendents, a program directory, through resource centers, in 
regional service center newsletters, and by presentation at state 
conferences. 'V, 

^ Finally, programs are revalidated* for two years following 
documentation of successfu4^ evaluation results^. 

^Contact Person * . * , ^ ' 

' \ )larj Wigfftman 

^ Director , ' . . ^ . 

^Division of Disseminaton 
Texas* Education Agency 

201 £• nth Street, . . ' i. ' . 

Austin, Texas 7^701 ^ ^ - " . 

. (512) 475-5601 • ' ' , ' / < 



/ * • * * - 

^ • '157 ^ , 

ERIC 167 - ' ' 



7 



UTAH 



Validation Process and Criteria Used 

The Utah State Boartl Education has. adopted a procedure which 
allows for three levels of endorsemfeot of educational projects or 
programs: (1) full endorsement; (2) provisional endorsement; or (3) 
endorsement as. a model program. 

• ^ • 

In order to be. fully endorsed, a program must meet three conditions: 
(1) it must be shown to be effective in producing ch^inge irf the target 
population; (2) it must' be reasonable in cost and have estimate^ of costs 
to districts wishing to adopt the prograqi; and (-3) it must be \^ 
exportable. JDRP approved programs are, automatically fully endorsed. 

Pro^visional endorsement means that: (1) the program is likely to be 
fully endorsed only if some additional, specified, product development 
and/or evaluation is carried 6iit; (2) it is believed that- dissemination . 
of the program would benefit le^irnens; and (3) the desired product 
development and/or evaluation processes could be carried out in. the 
developing and/or adopting districts in three years or less.^ Model 
programs must meet at least three conditions: • (1) the practice can be of 
value to other school districts; (2) at least drte LEA must agree to serve- 
as a demonstration site; and (3) the respo>isibilitlBs of that LEA;nust be 
agreed to in' writing. . • \ ; , ^ 

For any of these levels, an SEA .professional' staff member must ^ 
nominate the program. If a project or^ program is recrfmrrended for full or 



provisional endorsement, then the SEA advocatfe and 
the Office of Research and Planning assist the LEA 
application for validation. An external evaluation; 
application and recommends action on endors^ement. 
endorse or reject -the program is made'i)y th,6 Offic/ 
Services Administrator's Coordinating Commi.ttee. 



epresentative from 
completing the 
evjaluates the 
cision to . 
of 'Instructional - 



Training Procedures fQr> Validators ^ 

Currently, validators are certified by National trainfng programs. 



fa 



VaTi/lation Procedures 



•0 



The state uses, both in-state and out-of-state" validation teanjs in 
'Order to have several options available to project developers and state 
administrators. Validation teams for business practices consist of . three 
specialists covering the follovying three areas of concern: 
effectiveness, cost statement an^ efficiency, and exportability. 
Validation teams for other practices have expertise, in effectiveness, 
cost, and exportability. On-site visits are usually two days in length 
*and are usually conducted by three team members. ' " 
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UTAH (continued) 



Or^nlzational "Arrangements 

No S^tate D€?partment of Education personnel are specifically assigned 
to coordinate validation activities* Approximately three programs or 
pr'actices^are given full endorsement each year. Title IV-C developmental 
projects are encouraged to pr^are for validation in their second year so 
that problems carr be remedied by the third year. In the case of business 
practices, some projects that are not validated are given assistance in 
^freparation for validation. The "provisional" endorsement is a jpechanism 
used to assist local education agencies with the validation^ process. No 
specific schedule for validation is set by the state. 



Support and Dissemination 

Endorsed programs receive technical assistance from the state to plan 
for demonstration and dissemination. For business practices, the 
preparation' of .validation applications is facilitated by an educational 
research firm through a contract with the^tate. Adoption and adaptation 
activities are supported by Title IV-C grants. ^ 

Business practices are disseminated by the state at the state and 
national AS80 conferences. Other validated practices are disseminated ' 
through meetings of project directors of experijnentaT programs. 
Dissemination also takes ptace through distribution of Educational 
Programs That Work and through funding of adoption projects by Title 
' IV-C.^ in January of each j*ar, a listing and description of all endorsed 
programs is published. Developers of validated projects, with the help 
of Title IV-C grants, fare responsible for active dissemination of their 
project . 



Contact Person 
Or. Richard Keene 

Specialist in- Research" Design and Statistics 

Evaluation and Planning 

Utah State Office of Education 

250 E. Fifth South Street 

Salt Lake City, Utah 84Tir 

(801) 533-5888 
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VERMONT 



Validation Process and Criteria Used 

The 1980-1981 school year is the pilot year for Vermont's state 
validation model. In the pilot year, the process is considered less 
rigorous than either the JDRP or IVD validation models. However, the 
Vermont model does incorporate several effectiveness and exportability 
criteria common to the JDRP and IVD models. One unique feature of 
Vermont's process is the stipulation that all projects seeking validation 
must have been adopted arid funded by the local school district for ^t 
least one year after 'federal funds ceased. 



Training Procedures for Validators 

Formal trailing procedures have not yet been established. However, 
state staff and outside consultants skilled in both evaluation and 
dissemination are involved at each step of the validation process. 



Validation Procedures 

Several steps are employed in Vermont's validation process. ' 
Initially all Title I ancj Title III/IV projects are screened by state 
staff according .to a checklist to determine whether or not projects are 
suited for validation. At this stage five criteria are employed: 
funding level, local continuation, availability of ^ impact data, 
availabilijty of exportability infbrmation, and perceived statewide need. 

, Projects meeting initial criteria are scheduled for pre-validation 
site visits by state evaluation and dissemination consultants. Also, ^ 
state staff contact superintendents and project directors to confirm 
their willingness to become a state validated project. During 
pre-validation vls^its, state staff utilize two checklists covering a 
number- of effectiveness and exportability -criteria. 

At the conclusion of the visit, validators meet with local officials 
and either recommend a validation .visit by an outside consultant- or help 
the project develop a plan of action to strengthen their chances for 
future validation. 

Projects confirmed by a pre-validation visit then are visited by an 
outside consultant who verifies the findings r^endered by state staff 
during the pre-validation visit. The reports of outside consultants, are 
examined by state evaluation «nd dissemination staff. State Advisory 
Council members, and the Director of the Division of Federal Assistance. 
This team renders a decision' whether or not to validate projects within 
one week of the validation visit made by the outside consultant. 
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VERMONJ (continued) 



, Organizational Arrangements x 

Title I and IV staff are involved in the initial pre-screening 
o process.. Pre-validati on visits are conductd by state dissemination and 
evaluation consultants. The proportion of' state staff time involved in 
the yalidatjion process is determined by the numbd of eligiblejrojects 
each year. To date, no projects have been validated, but four projects, 
recommended by initial screening have received pre-validation vistts from 
state staff and are beirig scheduled for vjilidati-on visits by outside 
consultants. , • ^ . 

Projects recommended for further development in the pre-validation 
visit must apply for additional* support from Title IV by May 1st of each 
year. Once the additiorial development suggested by the pre-validation 
visit team is completed, projects are granted state validation.'- 



Support and Dissemination . „ 

All validated projects must be those developed under Title I and 
III/IV funding. Once validated projects are maintained oh local funding 
sources. Title IV funds are no!$ directly available to validated 
projects. However, Title IV adopter/adapter (A/A) monies are used to 
defray the developer's costs in training adopters and install ijig the 
project in the adopter site. x ■ 

Vermont A/A grants are usually "within the range of $3,000-J8,0C0 with 
an approximate average of $5,000. Districts adopting state vaTldated 
projects must apply in a competitive grants process for support. The. • 
duration df A/A -grants is one year. ^ 



. ' State-validated projects must agree to participate actively in the 
dissemination of the project. Validated projects 4nake' commitments to 
traift all adopters and otherwise assist in the installation o| the " 
project at the adopter site. 

, State information about validated programs is disseminatied via 
oatalogue^,^ direct contact with districts, brochures, awareness 
conferences, and a computer searcfi^f ile' system. 



•The Department also has a promising practices file Rnown^ as-VERB 
(Vermont Educational" Resource Base) in which Department of Education 
staff and practitioners in the field are encouraged to submit Vermont 
originated or Vermont developed products, publications, practices, 
methods, consulliants, etc., for entry in theWERB' data base. The 
accepted entries are put on master microfiche and made available without 
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VERMONT (continued) 



charge to elementary and secondary teachers on duplicate microfiche or 
" paper copy. The VERB entry abstracts are periodically issued in indexes 
And widely circulated through the State. > 



/ 



Contac^ Person ' 

Ronald Rubin • 
Coordinator, Title -IV-C 
Si^te Department . of Education 
Montpelier, Vermont 05602 

(802) '^28-3124 
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VIRGINIA 



Validation Process and Criteria Used 

Virginia uses the IVD process ^ind the validation criteria outlined in 
the Sharing Educational Success manual T ^ ^ 

Tralriing Procedures for Validators • ^ * ^ ^ 

To date, all validators have teen, trained by the U. $• Jtepartment of 
Education. <^ 



Validation Procedures * 

^ T^e validation j:eam is composed of thr^e people: - a chairperson, an 
effectiveness/success specialist, and an exportability spfecialist. 
Before 198> all validators were from out-of-state, which increased 
validity and reliability of findings. as well as precluding built-in 
prejudices* Beginning in 1981, teams have been composed of in-state and 
out-of-state validators. A two-day site visit is included as part of the 
validation process. 

Organizational Arrangements ^ ' 

Three people at the SEA are responsible for validation. ' All Title 
IV-C projects in their last funding year are given an opportunity to 
apply for validation. The SEA staff reviews and analyzes the readiness 
for validation of the projects. Those judged ready are then reviewed by 
the validation team. On the average, two or three projects each year are 
validated. * , • ' - 

Title IV-C staff provide technical assistance to projects desiring* 
re-submission for validation. Similarly, technical assistance is 
provided during the initial application period. 

While the schedule for-vaTldatlon is not fixed, a typical -schedule 
includes these steps: 

tetters forwarde^to all terminating projects in early 
February. • 

Desire of locaV division to be considered for y 
validation submitted t^tate by March 1. ^ 



4 
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VIRGINIA (continued) 



Conferences held with each submitter starting the end 
; of March. Title IV-C staff members and appropriate 
content personnel of the Department participate in 
these conferences. 

^ Those selected to proceed, i.e., those who have been 
identified as meeting the requirements, will submit 
' their validation proposals during April. 

Teams identified during April and on-site visits hfild 
in May and/or the early fall. 

Teams, during exit interviews on site, announce 
success or failure And submit their written report to 
the state Title IV-C representative at that time. 

Following^ review of the repdrTand the «/ 
recommendations, the State tepartment officially, 
notifies the super intendentNconcerned and forwards a:? 
grant awarjl for the firjt year of operation. . 

All state validated projects^^re contained in the ■ 
* Profiles and are Encouraged, if data are acceptable, 
to proceed for JDRP approval. 

Support' and Dissemination 

« • 

Projects which are state validated are provided up to $25,000 for two 
years to support disseminatiori-diffiision activities.' .The school district 
maintains the demonstration sit*e out of local funds. Adopter programs to 
install one-year-pilot projects are eligible for maximum funding of 
$10,000 to be used for training and materials. However, personnel 
•salaries, and capital equipment cannot be purchased from thiSu-mor 

P/Ds are eligible for grants to develop materials and ^o 
adopters. * 

The SEA also disseminates ihformatiorv- about val1(/ated progr 
through the Profiles Handbook and through staff recommendations 
potential adopters. 
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VIRSmiA (continued) 



Contact Person 

COL Francis T. Phillips, Jr. 
Coordinator of Special Programs 

and Grants 
Virginia Departfnent of Education 
Box 6Q 

Title IV-C Office 
Richmond, Virginia 23216 

(804) 786-3178/7215 
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.WASHINGTO^I 




compatible with the JDRP process, except Jhat the state requires on-site 
project review. Major Criteria used by validation teams include: 
evidence of a real need for the program/project; review of project 
objectives to ihsufe that they follow directly from the stated needs and 
are measureable; review of the evamation design to insure that it will 
'produce evidence of project succ&si or failure; indications that tbe 
project will be relevant and of interest to other districts and th^t they 
would be able to afford it; and review of the project to insure that it 
addresses an important problem in a meaningful way. 



Training Procedure for Validation 



Washington currently has 15 trained and certified validators. 
Certification requires two days of training plus involvemeat in on-site 
evaluation experience. Training fs. conducted by teams of three people^ 
who assisted staff of the Office of\he Superintendent of Public 
Instruction (SPI) in the development \of the state validation manual and 
who have been involved in 'the develofnwnt of Sharing Educational 
Success^ Once validators have been cek:ified, recertlfication Is planned 
for every, ttypee ^ears. Additional validators are chosen from a list of 
people inviTCd to send resumes for review. Twenty-three of the 64 
persons subn^ttlng resumes havie been selected and are called "Members of 
Tri-Partite." 



Validation Procedures j 

In putting together validation teams, major emphasis is placed m 
evaluatipn t)^a1iiing and experience. The team leadeV" must have high]^ ^ 
developed skilk jh(i^ evaluation and measurement, and other team members 
must have a solnd background and experience in curriculum development and 
administration/budgeting. On-site visits are conducted by threerperson- 
teams and ar6 of one day's duration. 



Organizational Arrangements 

The Director o^ Grants Management at 'SPI assumes jbh^ responsibility 
of coordinating the staters validation process. Washington has had such 
a process for one year, during which ten projects were considered for ^ 
validation and six were validated. Applications may be submitted at any 
time during the year# They^are reviewed at SPI for cdmpleteness and 
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WASHINGTON (continued) 



accuracy, after .which a decision is made as to whether to have an on-site 
team review the project. If the application is approved, the team 
conducts its review and prepares a recommendation. If the application is 
not approved, it is returned to'.the submitting district with suggestions 
)r improvement. Technical assistance is provided subsequent to 
ibmission of a preliminary application for validation. 




Support and Dissemination 



SPI provides fourth and fifth year funds for replication from its 
Title IV-C budget, generally in the range of $10,000-$25,000 per year. 
In FY 81 approximately $500,000 in TUle IV-C funds-were provided for 
adoption purposes, anc^approximately $1.2 million is provided annually 
from the state legislature for adoption. Career education and Title VI-B 
Handicapped also provide support for adoption. 



Information about validated programs and projects is disseminatied by 
the Washington State Facilitator, by^Project KNOW-NET. (the NIE State^ 
Capacity Dissemination Program) through Your Public Schools (the SPI 
publication with a circulation of 35,000), and by brochures; newsletters, 
and program and'SPI staff. 

Developers of validated projects are responsible for securing a 
minimum of one adoption per year, and preferably more thalh one. - Training 
and follow-up are provided to adopting districts. Project- staff make" 
presentations at education fairs, are available to provide project 
information, and disseminate project materials. 



Contact Person 



Jean Wieman 

Director, Programs /and Learning-Resources/Disseminatibn 
Office of the State-'Superintendent of Public Instruction 
7510 Armstrong Street, S.W. FGll 
Tumwater, Washington 98504 . ' . 



(206) 753-6717 
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^^ WES? VIRGINIA 



VaTidation Process and Criteria Used \ . f 

' ^ \ ~ ^' 

' A state developed process for validation of^education^il 'programs and 
practices is used. The review team Ipoks for evidence of educational 
effectiveness a»d" export ability. 

Training Procedures for VaTidators . " / 

An orientation (one hour) is conducted for team members in advance of 
the site visit, . ' ^ * 



Validatiort Procedures 

\ ' ^ — ^ s " ' 

The, validation team is composed of fivSoPeople: a member, of ESEA IV^C 
/Advisory Council^ an SEA content specialist, a higher education 
/specialist, a local education agency contfent specialist, and an SEA* 
evaluation Specialist, This combination was chosen because of^ 
coSt-effectiveneSs and because^ requir^emehts of the state va,1fdation 
plan. The members of the team visit on site for th^e ]to five houcs. 

Organizational Arrangements ^ .\ . ,^ 

Three staff persons from the SEA Title IV-C office are responsible 
for the coordinatio^h of validation actfvl^ties. They are assisted by. 
othePv^EA staff aS' required by. the sta^te procedures manual, ^On the , * 
averageTHen programs annually are considered for validation, of which 
two are va^li dated, * ' > 

Projects in the^j^hird fiscal. year are eligible for validation team 
review,'; Deadline for eiiplicatioru is February 15, A review of the 
application is made ty jMarch 15, On-site visits OQCur between May 1 and 
June 15, with applications for D/D grants due from successful validated 
projects by July 1; * - 

Support and Dissemfhation . , 

During the first year as a validated project (fourth program year) 
the project is eTIgitle for 75 percent of its third-year budget, not to 
exceed $25,000, In'the second year as a validated project, 50 percent of 
the fourth-year budget— tb a maxjmum Of $12,500— is available. Adaptors 
are able to compete for IV-C funds for amounts up to $5,000 for up to 18 
monthls, A portion'of the D/D awar%>is designated specifically for 
dissemination activities, • * * 
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: . WEST VIRGINKT (continued) • 

In addition, the SEA disseminates infomation about' validated 
programs through awareness conferences- and a variety of print media. 



' Contact Person 

Dr. Alpha E.< Wilson 
IVrX Coordinator ' 
West Virgtitja St'aiie Department , T, 

of Education 
Butlddhg 6',,»Room B-261 
Capitol Complex 

Charlest(Sn, West Virg/inia 25305 ^ 
(304) 348-2699 



• ' WISCONSIN 

Validation Process and Criteria Used . - ' 

Wisconsin uses the IVD process and the validation criteria outlined 
in the Sharing Educational Success manual. 

Training Procedures for Validators 

% Validation team members In Wisconsin received a one-day workshop " 
based on SES materials and Atlanta Region materials for the U.S. Office 
of Education. 



'Validation Procedures 

: ' Wisconsin uses three-person validation teams to conduct one or two 
day site visits of projects. The state uses only In-state validators 
primarily for cost considerations. . The teams consist of a content 
specialist, a person with evaluation expertise, and a person with a 
generalji.curricu'lum background. All team members rat? all aspects of .a 
project (i.e., effectiveness/success, exportability, etc.). 

Organizational Arrangements 

.One full-time and one part-time person- are responsible'for 
^1f(lsConsin*s validation program. Approximately five programs are 
considered for validation each year and, on t^e average^ three are 
validated. 



sd. 7 

statJe's sch 



The statJe's schedule for validation is listed below: 

Third year ^ Letter of. intent prior to application 

Summer after third Applicatipn" 
c. ^ year . " ' ^ - 

September-October . Review of the prtjject; immediate 

^ notification from on-site validatot^s 

January Applications for dissemination grants 

* • consirferad 

// 

State staff provide ^general con^lting assistance with project data • 
analysis; interpretation, and reporting. Data organization seeirts to be 
',ttie most cdnirnon problem^, ^ - . ^ 



• > «3 
WISCONSIN (continued) " - ^ ^ 

Support, and Dissemination ,^ 10 - • • 

.Validated projects are eligible for one yeau^'D/D grants up to 
$15,000, and are e^liqible for renewal upon application. Support of 
adopter/ adapter (A/A) activities ts a^ajor priority. About half qf the 
state nwnies HV-C) are used to support A/ As who can qualify for grants 
ranging from J3, 000 to $25,000. The average A/A grant runs about $18,000 
.to $20,000. ' . ^ \ 

Developers of validated.projects must agree to continue the projjBcts 
"anjj provide general "information to A/As. • 

. The state disseminates infofmation about validated projects through . 
Tijtle IV-C newsletters, regional meetings, and brochures. 



Contact Person 



Bill^Boviman/Nancy Scott 
Coordinator, Title IV-C 
Wisconsin Department of Public Instruction 
J25 S. Webster Street 
Madison, Wisconsin 53702 

(608) 266-5357 
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WYOMING 

Validation Process and Criteria Used V , ' 

Wyoming uses the JVD process and the validation criteria outlined in 
the Sharing Educational Success manual . 

Training Procedures for "Validators . ' ' °' 

Validators are trained at RotP workshops and/or state train.ing 
woi^shops. . ' 



Validation Procedure ^ ' , . 

The validation team is typically composed of three people: a /. 
research specialist, a content area specialist,^ and an administration 
specialist. Of the three, two are usually in-stat^, with the team /deader 
being brought in from^ another state. ^ . . / \ 

Applicants are prescreened.^ The site, visit isv a inajor.part q(f the 
validation process. All three team menbers spend three, days on-^ite; ^ 

Organizational Arrangements ' - ^ ' - . ^ 

One person In the SEA is primarily respt)n5ible for validation, 
although any trained consultant at^he SEA can work" on validation. T 

Since this, is the first year in whkh IVD has been. Used, it is . 
difficult to report numbers of programs cqnsidiered foV validation. To. 
date, three programs have been validatetf> 

There is no fixed schedule' for validation. Usually'a request is ^ 
forwarded to the State Superintendent from a local spperintendent. The* 
validation procesfs then \t set in motion. After t;he site vl5it, a 
program which \i not recommended for .validation is informed bf reasons 
^why validation is withheld. No formal* mechanism, however, exists for 
helpinjg programs which have not been validated. / ^ 

Support and Dissemination , > . ^ 

The state '-support of D/Ds is individualized/ for each project^. Level . 
of funding and sdurx:e of funding vary. The D/0 is responsible for 
developing awareness materials and making workshop presentetions. 
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WYOMlte (continued) 



'The stat6 supports A/As via materials and workshops. The state does not 
currently fund adopters but is looking for ways to 'support A/As through 
funding. ^Dissemination by the state .is handled through a state 
dissemination network, newsletters, and workshops. 



^ Contact Person , 

nr. Jack Prince . * 
■\ Coordinator 'of Educational Innovation and Media Services 
Stafe Department of Education 
l^athaway Building- ^ 
Cheyenne, Wyoming 82001 \ ' . 

(307) 777-6252 
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fIST OF STATE CONTACTS 



. Alabama 



s 



Or. R. Meade Guy 

Dir^tor, Alabama Information and 
Development System 
,, Room 607, State Office Building 
Montgomery , • Al abama 36130 
(205/832-3138) 

Al-asl^a •* 

*^Eula Ruby, Director 
Division of Education Program Support 
Alaska State Department of Education 
State Office Building, Pouch F 
Jui^eau, Alaska 99811. 



(907/465-2830) 
Arizona 



Dr. Beverly Wheeler 

Director of Information Dissemination 

Arizona State Department of Education 

1535 West Jefferson 

Phoenix, Arizona 85007 , 

(602/255-523i| _ 

Arkansas , 

* 

Mr. Fay Bofiannon 
Coordinator, Title- IV, ESEA 
Divisior^of Feder^H Programs, Title IV 
Arch Ford Eduction Building 
.State ■Capi.to,l/>1a 11 ■ ^ . 

Little' Rock, \rkansas 72201 
(50.1/371-1 245 )\/ 

Cal ifornia 

Ira D. Barkman 

Coordinator, Exemplary Prograqijj 

Replication Unit 
California State Department of Education 
721 Capitol Mall 
Sacramento,- California 95814 
(916/322-6797)- 



Colorado 
Chuck Beck ' 

Director, Colorado State Facllltatoji^ 

Project ' J,. 
830 S. Lincoln , " 
Longmont, Colorado 80501 
(303/772-4420) 

Connecticut 



and 



Br. Roger Richards 

Bureau of Research, Planning, 

Evaluation 
State Department of Education 
Box 2219 

Hartford, Connecticut 06115 
•( 203/566-8250) 

Dr. Richard E. ^Lappert 
Coordinator, Title IV-B-C 
State Department of Education 
Box 2219 

Hartford, Connecticut 06115 
(203/566-3593) 

Delaware 



Dr>. Peter J. Bachmann 
State, Specialist, Educational 

Facilitator and Program Dissemination 
Delaware Department of Public. 

Instruction 
P.O. B.<5x"^1402 
Dover, Delaware 19901 
(302/736-4583) 





Dr. Ralph (f. Vedros 

Program Administrator 

Department of Education 

Knott Building i ^ 

Tallahassee, Florida 30301 

(904/488-7128) 



8/81 
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Georgia - 



Or. 'Je§s Pat' Elliott 
Oir^ctpr of Research 
State department of Education 
Atlanta, Georgia 30334 
{'404/656-2402) 

Hawai i 

Winona Chang/Kathleen Steffen 
Office of Instructional Services 
Hawaii State Department of Education 
233 South Vineyard Street 
Honolulu, Hawaii' 96813 
(805/548-3425) • 



Idaho' . ^ 

Charles Binowi 

Coordinator of Dissemination 
Idaho State Department of Education 
Len B. Jordan Office Building 
Boise, Idaho 83720 
(208/334-3225) 

Illinois • . ' : 

Karol Richardson 

Assistant Manager, Title IV.- 

Illinois State Board of Education 

100 No. First Street 

S^ingfield-, Illinois 26708 

(217/782-3810) 

Indiana 



Don Treibic 

InilianateOepartment of Public Instruction 
Division of Innovative Education 
Room 229, State House ^ 
Indianapolis, Indiana 46204 
(317/927-0386) 



Iowa (continued) 

Max son. Chief" 

ESEA Title IV and -Dissemination Section 

Towa State. Department of Public 

InjStructibn 
Grimes State Office Building 
Des Moines, Iowa 50319 
(515/281-5274) 



\ 



Kansas 



Phil Thomas 
Director, Educational Improvement 
Section' 

Kansas State' Department of Education 
120 East 10th • 
Topeka,' Kansas 66612 
(913/296-3136) " - 



Kentucky 



-Nil 



Geri Weaver 

Director, Division of Research and - 

Evaluation 
ffice of Research and Planning " ~\ ' 
Kentufcky Department~pT 'Education ' • 
Capital Plaza Tower 
Frankfort, Kentucky 40^01 
,(502/564-4743)... * . ■ . 

Louisiana 

Sue F. Wilson 

.Section. Chief , Dissenij nation 
Louisiana Dissemj nation Network (R&D) 
Bureau of Development 
Louisiana Stafte "B^artnient of Education 
P.O. Box 44064 
Baton Rouge, Louisiana* 
(504/342-4268 or 4269) 

Maine 



708^ 



Iowa 

David Lidstrom, NDN State Fac. 
Iowa State Department of Puyb 
Grimes State Office Building 
Des Moines, Iowa 50319 
(515/281-3111) 




or 



> -^Dr. Lois W. Jones 

Title IV-C Coordinatbr . , ' ■ 

Department of -Educatfonal and Cultural 
Sp-ience ' 
'struction Stat|bn 23 ' 

. Augusta, Maine 04333 
(207/289-2475) 

■ " ■ • • ■ '878,1 



178 



i86 



Maryland 



Montana 



John A. Johns 
' Specialist in Program Assessment 
Maryland State Department of Education 
200, West Baltimore Street 
Baltimore, Maryland 21201 
,(301/659-2380) J 

Massachusetts 

Dr" John Reynolds ^ , 
Massachusetts Coordinator, Title IV-C 
Springfield Education C^ter - 
■ 155 Main Street • • v • 

Springfield, Massachusetts- 01105, . 
(41-3/739-7271)/ . - ^ . 

Michigan » .... 

John Osborne • ^ 

Coordinator, Experimental' Demonstration 

Center Programs 
Box 3d008 

Lansing, Michigan 48909 
(517/373-1806) ' .•• 



Minrresota 



Len Nachman 
Evaluation. Supi^tivi^pr 
Minnesota Deparment of Education ^ 
Capitol Square Building ^ 
550 -Cedar Street • 
St. Paul, Minnesota 55101 / 
(612/296-5032). 

I • * 

Mississippi' 

Clyde Hatten 
.^ Coordinaftor-ESEA Title IV ' ■ • ' ' 

V Missis/ippi State Department of Education 

' ^ P.O. Jox 771 

Jackson, Mississippi 39305 0 t 
(6,01/354-6958) :^ ' < • 

Missouri . * 



.Tom Odneal 

Director, Title IV-C • 

Missouri Department of Elementary and 

Secondary Education - 
■Box 480 

Jefferson City, Missouri 65101 
(304/751-3468) , ' 



ERIC 



Judy Johnson, Assistant Superintendent 

for Special Services 
Office of the Superintendent of 

Publ,ic Instruction 
State Capilot Building 
Hel ena , JHtana \ 59601 
(406/447^693) \ . - 

fybraska 

Jack Baily" 

Administrator -of Special Services 
Nettfaska Department of Education 
P.-O: Box 94987 
301 Center Mall Street 
Lincoln, Nebraska' , 68509- • • 
(4:02/471-24^,1) 



Nevada 
Al Ramirez 

Nevada State Department of Education 

Capitol Complex 

Carson City, Nevada 89710 

(702/885-5700) 



New Hampshire 



James V. Carr 
Coordinator, ESEA Title IV 
New Hampshire State Department of 

Education . • 

64 I^orth Main Street ^ 
Concord, New- Hampshire 03^01 
(603/271 -3481j 

New Jersey 

Ronald E. Lesher 
Director, program Systtems . ^ 
New Jersey' Department of Education 
Division of Operations, Research," 

and Evaluation ' 
P.O. Box 20 V9 
Trent^'n", New Jersey 
(609/292-7984^ 

j ' 



08625 



8/81 



187 



New Mexico 

Susan Brown - . . . v 

Coordinator, ESEA Title IV 

New Mexico State Department of Education 

Education Building, Room 102 

Santa Fe, New Mexico 87503 

(505/827-5441 ) 

New York 

Richard L. Egelston" ' ' ♦ 
Validation Coordinator/Title IV-C 

Program Manager 
'ftoom 860 EBA ' . ' r 

New York State Education Department 
Albany, New Y.ork 1?234 / 
(518/474-2380) > \ 



North Carolina 

Mr. Henry Helms ° ' 

Director, Division of Development* . 
North Carol'^na Department of Public 

Instruction 
'Education Annex I 
Raleigh, North Carolina 27611- . 
(919/733-7018) 

North Dakota 

Dr. David Lee 

Director, ESEA IV-C * ^ 
Department of Public Instruction 
State Cap'Miol 

Bismarck,' North Dakota 58505 
(701/224-2391) 

Ohio ^ ' \ 

George Harris 

Educational Consultant 

Division or Planning and Evaluation 

Ohio Department of Education 

Room 802 

Ohio Departments Buildipg 
65 South Front Street , " 
Golurtibus, Ohio 43215 i 
(614/466-4838) ^ 



Oklahoma 



^ Al ^Underwood 

* Administrator, Tit;le IV-C 
Oklahoma State Department' of Education 
Olivier Hodge Memorial Education 

Building 
2500' North Lincoln Boulevard 
Oklahoma City, Oklahoma 73105^ 
(405/52*1-3694) I 

regon • < * - 



Nelssp 

f;it>ecialist ♦ 
{SEAr^t\ttlejlV-C t. " 
)0 Pr\ng^ Parkway, S.E,, r 
Salem, Oregon 9731 (f / 
(503/378-6799) / f . • 

Pennsylvania ' - 

Waldo G. vreaver-y-Cdo«l^inator :> 
Pdnnsylva/fia Diffusion Network 
Pennsylvania Department of Education. 
Box 911; 333 Market Street ' - 
Harrisburg, Pennsylvania 18108 
(717/787-7372) ; v 

Rhode Islanr V ' 

^Dr. Uohh Custer 
Consultartt, Tit^e IV . 
State Department of Eckicatidn 
234 Promenade Street * ' 
Providence, Rhode Island^ 02908 
(401/277-2617) ^ ; 

South Carolina ' " ^ ' - ' 

Mr. Joseph C. Pate^ • 
Chief Supervisor, Title IV-C; Section 
Office of Federal Programs ' 
South Carolina State Department of 

Education ' • ' • 

1429 Senate Street - 
Columbian South Carolina 29201 
(803/758-291-1 ) 
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South -Dakota 

# 

Gene Dickson ' 

Oireetor, Title" IV-C and State 

Facilitator 
Department of Elementary and Secondary 

Education 
Kneip Building ' 
Pierre, South Dakota 57501 
(605/773-3395 y 



Tennessee 



3 




Barbara Oakley Adkisson 
Developmental Programs 
Tennessee Department of' Education 
116 Cordell Hull Build4ng 
Nashville, Tennessee 37319 
(615/741-1896) 

Texas- . • 



' Marj l^ightman 
"^^rector ' 
» DlA<:ision of Dissemination 
Texas Education Agency , 
201 E. nth Street 
•Austin, Texas 78701 I 
(512/475-5601) -. ^ • 

Utah 

Dr. Richard 'Keeije 

Specialist in Research Design and 

. Stati-stics 

Evaluation and Planning 
■ Otah Stiite Office' of Education 

2B0 E. Fifth South Street 

Sehlt Lake City, Utah . 84111 - . 
• (BOl/533-5883) 



1 



ermont 



7 ' 



. Ronald Rubin ' a. " 

Coordinator, Tftle IV-C ' 
?State Department of Education 

Ntontpelier, Vermont •05602- 

(802/828-3124) 



Virginia 

COL Francis T. Phillips, Jr. 
Coordinator of Special Programs and 
Grants 

Virginia Department of Education 
Box 6Q 

Title IV-C Office . 
Richmond, Virginia .23216 
(804/786-3178 or 7215) 

•Washington - * • ^ 

Jean Wleman 
Director, Programs and Learning Resources^ 

Dissemination 
Office of the State Superintendent 

of Public Instruction ' ' • . 
7510 Armstrong Jtbf^et, S.W. FGll 
Tumwater^ Washington ^8504 
(206/753-6717) 

West Virginia -^.-^ 1 



Dr. Alpha E. Wilson 
IV-C Coordinator 

West-Virginia State Depay*tment of 

Education 
Building 6, Room, B-261 
Capitol Complex 

Charleston, West Virginia 25305 
(304/348-2699) ^ 

■ 0 
1 

Wisconsin ' 



4» 



Bill Bowman/Nancy Scott 
Coordinator, Title IV-C 
l^isconsin Department of Public Instruction 
125 S.- Webster Street- 
Madison, Wisconsin • 53702 . 
(608/266-5357) 

Wyoming . . • 

Mr. Jack /Price ' . . 

Coordinator of Educational Innovation 

and Media Services ^ / 

State D^ar&ieot of Education v. ' ' ( 
Hathaway Building' 
Cheyenne, Wyoming 8200 T '^ 
( 307/7,77-6252 )■ . ' 

' ^ ~ 8/81 
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^ The^VD, JDRP, arid Sharfhg Business Success PrtJteSsesV 

X 

A Brief Explanation 



\ 



This material. was drawn from "The Search for Q&ality Control in Dissemination 
of Educational Products and Practices: 'AvLook at the Literature and Hajor/ / 
Issues," written by Linda Reed, R&D Interpretation Service^ CEMREL, .Inc.. , ' 
for the RiD txchange, 1981.' " ~ ' 
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The Joint Dissemination Review Panel 

'Almost 250 pr'dgrams have been validated since 1972 by the Joint Dissemination 
Review Panel (JDRP), originally the Dissemination Review Panel. In 1975 the 

. ' United States Office of 'Education, which originally developed the process, 
enlarged the" panel by the* addition ^pf representatives from the National 
Institute of Education and maden't the Joint Dissemination Review Panel. 
From 1975 to 19.80, the panel -was composed 'of 22 members, eleven from USOE and 

. 11 from NIE. Members of the. panel "were chosen for their experience in educa- 
tion and forrtheir ability to analyze evalirative. evidence on the effectiveness 

=• of. educational products anc^^ractic^s. . . • . 

.^RP review was ongir|ally conffned to programs developed with ^ederal^ funds'. 
The panej now reviews a" broad r|nge of prog»%ms' which come from all states and 
which 'hi^(e been devel6p6d^ with funds from a variety of .sources'.- Only proprieta 

. projects a^d prbd&:ts are npt eligible for review. . " 

- The Joint Dissemination Review Pan^l Ideabook , the major publication of the^. 
panel, establis+iesl^ an^r^xi>fertiw in deitail, criteria /f pr .evaluating "programs, 
giving sa^pl^'of convincing and noji-convincing data from validation ° , . 



appl-icat^s^ The Ide&book al.so lists typifal evaluation hazards which 
valida'tion teams should be alert for. " \ • > 

• ^ ^ / ■ " 
Six question^ must be'addressed by a validation team as it exarai;ies~«n,educa- 

tfonal practice, although the Ideabook, makes it clear' that evidence of 

"effectiveness is the' sole- criterion for approval by' the JDRP"!:' ' 




ERIC 



Tallmadge, G. Kasteh."- The -Joint Dissemination Revfew .Panel Ideabook . 
Washington, DX.; 'U.S. Office-of Education artd National Institute 
of Educ^tioa, 1977. ■' ' ■ ^ . 
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^1. Did a change occur? Was. the change a positive one that was in some 
^ - .way related to the objectives? . " . ' • 

2. Was the effect consistent enoi/gh and observed often enough to be 
statistically significant? - , * 

3. Was the effect educationally significant?. . . In judging the 

f - ^ educational significance of an intervention's iM)act, two factors ' 

must be considered: thersize .of the effect and the importance of the 
area in which it happene^. Thene^ought to be a reasonable balance 

The chance that a small gain would Be . ^ 
significant is higher Jn a broad or . 



between the two factors, 
considered educationally 



educationally ^important afrea than in a narrow or less important .area. 

4. 'Can the interventi^ be- implemented in another location with a *^ 
reasonable expectation of comparable impact? 

Is the project setting unique? 

Is the project effect Solely due to the untrue 'characteristics 
of the staff? 

What evidence is there to suggest that the intervention would 
work with different participants, in a different seating, and 
with a different' staff? 

WJiat components are essential? Can these be disseminated? 

5,. How likely J s it that the observed effects resulted from the 
intervention? 

^ Can plau'dible alternative explanations be generated? 
^ ^ Can the alternative explanations be rejected? 

6. Is the presented evidence believable and interpretable? 

. / Are there any Apparent inconsistencies in the data presented? 
' Are enough data presented to satisfy the skeptical evaluator? 
Are the .inferences drawn from the data consistent with the 

ev^idence? • ' . ^ ' * 

Has ,evidence been presented tha*t common errors have been . 
avoided? 2 ' 



Project staff who decide to submit their project for review by JDRP follow a 
specific format, for submitting materials (see page!), they can submit no 

J) ^more than 10 ^ages of explanation and documentation. The difficulty of| 
selecting the most appropriate nformatioa and of demonstrating avoidanc^of 
typical evaluation pitfalls leads most project directors to hire outsidg 

^ evaluators.^ 

^ - .. • 193. ■ . 




FORMAT FOR SUBHItrtNG 
TO THC JOINT OISSEMINATIOX REVIEW PANa 



Footnote 3 



^^^^^^ ^U^J rtidln^. ctrttr. tdtcatlon. tnvlrofMntil tducatlon, tducitlon fdr tilt 

u immjmoii title, loanoif: 

'sSltta?* ^ ^ inttnrtfitlofl th« location for which tvldtnco of tfftctlvafmss Is bting 



II. OCVElOra) BY: ' - 

i2^f2?J!j2i.^i22S ^Jiil^A^^"^. ortgintlly; .Ytn If thii bapponod at a difftt^t sita' than tho 
om for which avioanca of affactlvanass Is baing prtsantad.v^ 

III. SOURCE m LEva of FUNOINC: ' <; 

i:!!!^!^^^*"^""'^'* ^ Intarvtfitlon at tha location for which ayldanca of affactlvanass is 
pra$anta4 ind» for HCh sourca, list tha ammt of funds <saa Figura 1 for an axaapla). 

nr. YEWS OF IflTERVEHnOR OEVaOPMEHT: 

Micata tha yaar or yaars durtng which tha Intartantlon was originally davalopad or tastad.' 

y. wu oesauniON v nrotycxrioHt 

What is tha Intarvantton? 

What ara Its objactlvas? 

Miat clalas of aff^lvanass ara blno aada? 

What Is tha ^ontaxt In which It opmtss? 

Who art tha Intandtd usars and btntflclaHas? 

What art ^ charactaHstIca of tha groups on which tha Intarvtntlon wu davtloptd and tastad? 
What art tht sallant faaturts of tha Intarvantlon? v««»* 
What art tht castt of adoption and aalntananca of tha Intarvantlon? 

VI. EVIOOCE Of EFFECnVEWSS: 

2S^??J!?* ^ affactlvanass for tha Intarvtntlon. This saction should daal with aach of 

tht foHoMlng points, although -not nfcassaHly In tha SMt ordar: 

Intarprttthility of aaasurts* Svldtnea that tht quantltatlvt 'taasuraa ara rallahia and valid 
indiators or tnt 9ff%cxs clalaad. 

Sl!^^^!]^ 9^ Who cotlactad and analyztd tht data, what assurancas ara thtra that 

tnt findings arf oojaetlvt? 

£x!d^L4£Jsei$* tht avidtnca that sontthing happantd? What art tht afftcts. cUlatd 

foir tha Intarvantlon? 

.Ci{1danca Of <tat1st1q1 r|| lability of th^ «ff#i^. whaf is tht avidtnca that tha affaets happanad 
oftan anougn ana with suf ficiant raiiaAinty ta oa llkaly ta happtn agaln^yndtr slallar clrointancta? 

Erfdtnct t|gt tht affaets art adocationanv ttanlmrftil? what Is tht wi6mi^ that tht affaets ara 
53? SS3!!i^?y?^^?? ?^ y'^ fportant anougn to bt>aduat1ona11y aaanlngfuU rtgardltss of 



gfidanea that tht affteta ira attHbutafalt to tha Intarvnrtoitt Can altamatlvt axptanatfons > 
such u practict artacu, taturation, saiaction of suptHor traatMnt>groupst ate,, ba nilad out? 

IviaanS thartna*^^"^^ ^^^ oowlatlons for which tha 



ErTaanca tnat tht proouct.or practica has t>aan tastad w?daly tnough and Jndar siSflclantly 
divtrst elromtincas ta giva assuranet that tht affactr claltad «ay ba sitllar whan tha 
*: product or practica Is uaad alstwhara for tha populations Intandad, 



>_prodoct or orai 



!r^s3i 



If Intandtd ; 



popujatlons 
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.The review protess actually begins with the belief of a project officer that 
the project is worthy of national dissemination. All submissions- are reviewed 
in-depth by the originating federal program office. Trtey are examined for 
factual accuracy, social fairness, and posstble h^rm to usep, as well as for 
evid6nce of effectiveness. MTie final re^ew is cpnducted »by a seven-member 
sub-panel of the JDRP convened by j:he Executive Secretary. A vote is taken on 
the submission immediately after the review—a simple majority is required for 
a favorable decision. JDRP has been Icnpwn to ask for resubmission where 
evidence is less than adequate, for a decision. The minutes of ell reviews are 
available to project staff and the general public. 

Projects that are approved by JDRP become eligible for,' but are not 
guaranteed, dissemination funds distributed by the Department of Education to 
selected projects.. These projects become part of the National Diffusion 

* • 

NetwoVk .(NDN), a nationwide system established to assijst schools, 
postsecondary institutions, and others in improving their education pr-ograms 
through the, adoption of^xempTary education projects approved by JDRP; such 
projects are known as Developer/Demonstrator (D/D) projects. With these 
funds, and with the' assistance of State Facilitators (SFs)— offices located 
■with each state, and funded by the Department of Education to help local 
schools and others learn about and adopt D/D projects— the staff of validated 
projects can assist in the adoption or adaptation of their programs or 
practices at other sites. D/D projects are described in the NDN catalog, ' 
Educational Programs that Work , which is distributed nationally. 

» 

■ ^ ' : ■ ■ 
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J • 

Identification, V^lfdation, Dissemination (IVD) 

* • 

The IVO process was initiated jointly b/ the Tiational Advisory Council for 

Supplementary Centers and Services, State ESEA Title III Coordinators and ESEA 

Title UI personnel in the U.S. Office of. Education, and the National 

Association- of State Advisory Council Chairmen, with input and assistance from 

the State Departments of Education. Although IVD was initially designed for 

the validation of Title III (now Title IV) programs, the developers hoped that 

\ije process would be used for validating other programs as well. In fact, 

' most states that use the IVD process use Title IV fundf to validate programs 

4 

, funded under that Title. 

The IVO process represented for its. developers a quality control effort which 
would serve a dual function: 

First, it would provide a systematic mechanism by which the 
educational value and effectiveness of emerging programs, practices, 
and products could be reviewed and assessed as to their success; 
•. 'i.e., the purposes .and outcomes of developmental projects would be v; 
'■•"proven to work." Second, the validation process would enable thev • 
^ creation of a bank of proven educational practices. Once programs 
.and products have been validated as effective and exportable, they • 
could be entered into this bank of. successful programs. The 
- "validation program, therefore, addressed the need to identify and, to./ 
ceBtify programs and practices that could facilitate constructive ' 
e&cational changes in our nation's public and non-prof ft private 
schools,^ and provided guidance not only in establishing the 
educational w|orth of a project at its original site but also in its 
successful replication in other school systems. 5 , . 

e IVD process is guided by/a handbook called Sharing Educational Success; , A 
Handbook for Validation of/Eduimional Practices . The or-igipar handbook made 
%ery explicit the procedur^^5to be used by the state, by the va,lidation team, 
and by project staff. It o\fere^thr6e very general criteria, but the forms 

•:• • • 

t 

erIc ' ' ^•96 , ■ : 
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in the handbook , made very explicit the requimnents for meeting these 
criteria. Projects were given no guidance in Sharing Educational Success 
regarding evaluation hazards or kinds of data-that represent proof of 
achievement gains. 

In 1979, a revised edition of Sharing Educational Success was completed by the 

U.S. Office of Edu^rt^on in cooperation with the Association of State Advisory 

Councils and the State Departments of Education. Three major considerations 

led to the need for a revised edition: (1) the states felt the-^eed to 

develop a cadre of individuals at the state level who would be well trained * 

and who would ensure quality control of the validation standards nationally; 

' (2) 'there was a need for state autonomy in the validation process-^autonomy 

which would allowrthe states to modify, within federal guidelines, processes • 

, ^ / \'> - 

and procedures for project validation; and (3) there was a need to reduce the 

cost of the IVD process for both the states and the federal government.^ 



In the i7D process, each state validates projects that it believes are 
Exemplary, based on the following criteria: 

1. Effectiveness/Success: Project objectives identified for validation 

are supported by convincing evidence showing statistically and 

educationally significant outcomes. 

The documented effectiveness or success of a program or practice 
is of paramount importance for validation. A program or 
practice can be "proven to work" in numerous ways including: 
- (1) by demonstrating^with convincing evidence that the program 

will bring about desired change or improvement over* the existing 
. practices^ (2) by demonstrating a mrk efficient qr cost- 
effective program or practice' through improved management, 
resource utilization, etc., or (3) by demonstrating with 
convincing evidence that a desired objective may be accomplished 
without detriment to the existing program! 7 



2.- Exporjtability: Information is provided .to demonstrate pro^'ect or 
pract^e is capable of being diffused to other school districts and 
can be acjopted or ad^pt^d iy other school districts with similar 
' neelis and environments! 

/ 

For the project, as a whole (or for each 'applifcable component) 
information required includes evidence of educational 
significance, a descrifjtion of the minimum level of adoption or 
replication which would produce similar results, and information 
about the target population; staffing ^nd training requirements; 
materials, equipipent, and facilities; replication costs; and 
special .problems. « ' 

Sharing Educational Success describes six steps to 'be taken in the validation 
process! These steps can be modified by individual states to meet their own 
needs, and time- and money-saving options are suggested. 

1. LEA completes and submits application for validation to the state 

' . agency or office responsible fpr coordinating validation activities ' 
at the state -level. 

♦ 

2. Preliminary review by the state agency for validation followed by: 
, ' a. approval for validation team review; or 

, " b. return to the LEA for revisdon according to suggestions; 

^ Of' , • ^ 

c. disapproval for further validation. 

3. Selection of the validation team: 

■ '^r a.' the team leader to be selected from out-of-state from the 

list of USOE-trained team leaders within the' region, 
• b. two team members selected from within the state from the 

list of State-trained members. 

4. Review of appl icatibn by individual team members. {This may be done 
, as individuals in isolation from the other team mertibers or the team 

may be convened to*revtew an individual application or serve as. a 
panel to review a group -of applications): 

a. the team leader makes recommendations to the state agency 
, ' responsible for validation that thef identified revisions 

/ are to be'completed before the on-site team visit is 

condu'cted. 

b. the team leader informs the state agency responsible for 
validation that the on-site visit be conducted according to 

" * the existing application. 
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5. Conduct of the on-site visit. The most fy^equent procedure is to-send 
. alT^team members on-site. . Some states have designated an individual 

team member to conduct the on-site vtsit as a cbst savings ^measure. 
The individual conducting the on-site visit serves the role of. 
collecting and clarifying any incomplete or missing information 
identified by the review of the application by the whole team. 

6. The primary decision of the validation team is either approval or 
disapproval under the IVD standards, . p ^ 

If approved the team might also make the following recommend.ations: 
1. Submission • to ^JDRP 
, 2. State Dissemination 

, 3. A special component or product be recognized as worthy of 
distribution. ^ ' * • * 

; . , . • * ' 

An abbreviated .form of the application requirements fV IVD validation is on 

pages 9 and 10 J° ' The requirements set forth in the original edition ^f 
Sharing Educational Success ^ again in abbreviated form, can beVound in the 
Appendix. 

The IVD process requires no^commitiiient from the federal government, beyond 

assistance in training validation teams. After a program or practice is ♦ 

validated, the developers of Sharing Educational Success recommend that the 

state take one or more steps: * / 

Successful completion of the federal-state IVD process would be tied 
to a "pay-off to the project and to the state. -Such rewards may 
include, but not be limited to, entry into-a-s±ite-operated 
diffusion network, access to funds for project diffusion, ' 
preliminary screen;ing for national JDRP validation, or to otherwise 
' - serve as a vehicle for educational improvement within the state. H 

The Department of Education may furnish^technical assistance, for state-level 
diffusion, but only if a state-validated project is also approved by JDRP.' 
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ytrt I. Iwforwtlew ii4 avtrv1t» 

A. ADPllcint In formtiow (includlnrf io^itiiM\ ^ 

i. PrtiJtct Abitnet or Ovrv'lew ' / 

tl9«1f1cwc«^pf ruults. cost txporttblllty .ftctorsv vj»r^w^, r«iiiw, 

Part tl, gfftctlvtnit/SuceMi , . 



3. 

4* 
S. 



Idfwtify ttf tmior-paaou of tUt progrw or prtctlct^ 
9JI 5? jntldptttd chfngts or objtetlvts of tht pro^rMi or prtctlct, » 
ir^-^lz how iwch ciwugo In promrv bttavlor wu txptcttd for ttdi objtctlvt if this ms net 
1nc1u4td in tht stataMfit of objtctlvts. m • 

OtscHbt how tht Mjor objtctlvts art inttrrtUttd and if thty art of tcnia^ iin»rtanct, 
Id»t1fy mm or unantleipattd objtctlvts u wtll as any objtctlvts that Mtrt dtUttdNturln^ tht 



8. ^^ooraw Actlvltits 



Otscrtbt t|M procass(ts) including tich kty titwtt, such as: 

a. Mat tht Itt'mtr did dlfftrantly . 

b. Uhtt tht ttachtr did difftrtntly 

c. Ust of traditional or noiKtradltlonal MttHals 

d. Sptdal mtgtnnt ptan(s) 

t. ftirttlon and Inttnslty of proctss (l.t,. dally schtdult) 
f. InvolvtMRt tf partnu and/or 1:11— imfij 



C. gvtluatlow tWtl^ 
1. 



2. 



Otylbtytfly tht tyalttatlon dttlgn utlllitd In tht projtct, {Tim strits,. bastlint, nom 
rtftrtnctdt traditional t ip trl t ntal^control dMiyn, discrtpancy ■odtl. east stiidy. ttcj 

i^'^wT!!?;!!?' 1wtniiitnts,or dat^thtHng ttchnlws utillxtd Mrt valid* rtliablt 
and stnsltlvt.A.Tht fdlloidng foimt-ls suggtsttd for Hch InstruMtntt 

a. Ttst or data gathtrlnedtvlet \l ' 

b. Validity » >^ 

c. Rtllablllty * 

d. Ntnf group (If non^rtftrtnctd tast») 
t. CHttria Itvtls (if crlttHon^rtftrtnctd ttsts) 
f. Othor rtltvant eharacttrl sties 



I* 



ihm that avi dtn ct wcs sys 
ttsMlts and /Mialvsls 
1. 



itlcally gathtrtd and rtcordtd. 



2. 
3. 



teport tht ftsalts of tht proctss inttrvtntlon. Rtlatt thttt rtsults to sptdfltd objtctlvts. both 
proctss and product. Indlatt Wwthtr rtsultt not or varlid frm txptcutloM. ^•^^^^ 

Tht*folloiHng ^orMt Mtvld bt htlpful In rtsponding for tach.objtctlvt: 

a. Exptcttd changt j^r antlclpattd outeopt 

^* iJ^L^**^ pr-rtsnlts. Otilirt charts, graphs* statistical suMsHts Mhtrt appwpiHatt 
c* Slfnlffeanot of rtsults-^lthtr statistical or othtrwist. If othtr than Statistical providt 
ratlontlt fd^^mluttlon of signlflcanct. 

Forovtrtll projtct rttttlts: 

a. Cstlaatt of adocatlontl or practlcal slgnlflcanct of findings 

b. %ri9f intarppttatlon of rtsults 

Show that tht rtsults i«rt systtMttcally and coi^tttntly analynd. 
tonort onantlclpattd outeoMS of najor iapbrtanca and signlflcanct. 
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ZYO SUtmSSION REQUIRIHEXTS 
(Abbrtvlatad Fom) 



Hn rn> ExocrtJbllltv V 

Mtanais. This sftction irfU identify and (toeuMnt ttit vr^rm^t capability to7iffuM a sucms^l progrM. 
U .EducatKmal Slanifii^ ' 

SSS'^SA^t?^ c(«wjlty, MgnitiMlt of the problaa, bti^lts of a rtp11cat1«i In 

' 2. Tarqtt Pooulatlon ^ 

!Sri*^L^*!IT WUtion for tht rtpl1ea$1on of tto progrw and any uniquo « 
cfcartctw^itlci about tba oHginal sita that My lT«it ttit sueeasa of a rJSlicatlon*, ^ 



3. Staffing and Tr^ittiiw Raouirt— it^ 



ittff utuatly mn«6l« tol sOmmI district; can tiiT^liiln, 'b« s^wtt«17 " 
mttrlili. EaulCBWit. FieWltiM 



SSS^I? *ilJ!!2II^'Ti*'"°?^ ■«t«^«ls. aqulPMnt. <nd ficllltlM iMcutary to rvpllata tiM 
progns. Prwrtdt eofflts iurint on-«1ta visit. 

5. mimm Adootlwi or «»»11eit1ai ^ 

' S2S2?,J2lfJ!^*,2!!r**^ • t"^— ^*r*' ^ rtpJIatlw or pro^ra* that wvld product 
SSlStJS^ ^ «tocaw«t«d at sueeassful. Can Imtlvlduit canpoMots b«^^ 

I. tapHcitiow Coata J 

" . Oatail all com, ineliiding casta of traininf, wtarials, and start*yp. 
7.^ >oac1aI PrabloM 

SSi? o^D^cS J;%'2d5' ^ "d oparatlon of ^r 
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Sharing Business Success (SBS) 

• f % 

Business practices which successfully go through the procedure outlined in. A 
Handbook for Identification, Validation, and Dissemination of Sdioo'l Business 
Practices receive validation approval from both the* State Education Agency and 
the Association of School Business Officials. The .Process is similar to the 
Identification, Validation, Dissemination (IVD) process, in operation but is- 
simikr to the JDRP process in its rigor. 

- ■■■■]>. 

Applicants for validation are require'd to subm.it a preliminary application 
which is reviewed by a screening committee. Applicants supp^ly the following " • 
information: ^' (1) a statement of major outcomes in objective form; (2) a. 
description in sequence of the changes, interventions, or activities whch 
caused; the accomplishments or outcomes; (3) a description of how the outcoitfes- 
were evaluated and evidence that achievements were significant, the result of 
the new practice, and persistent over time; (4) a description of materials 
and/or 'equipment used by the practice; (5) a statement of all costs; (6).'a 
description of savings in time and/or money; (7) a description and humber of ' 
pei^sonoel directly affected or involved in the practice; and (8) a statement" 
about other relevant information or special conditions bearing on the success 
•of the practice. ^2 . . . . ^ 

Those local education agencies which are invited to make full application must 
supply extensive i^nfonnation about the practice—its effectiveness, ^ 
. efficiency, cost, and exportability— and must undergo at least a one and a 
h^^lf day site visit. ,The criteria for s\jccess are those used by the Joint 
Dissemination Review Panel .(JDRP). * • 
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»MtIM RSIIKSS SXaSS 
• Subrtstlon RtqulriMrtts 



I. Itfic tnfofwrtiort 



III. tff«ert^>«i>«i crtt«H<» ' ^ / ^ 

Japply OM'^ltoMliif InfOifMtlM for Mcii'objaetlvt: ' ' 

I, ItmtAtt 'wA. (iMerlbo Md^ \wrmmi or liMterlnf tfayietrtait la tin tMluttloo.' • * 

I. W** ovt^onea tint porsom mpamlblo fOr data caltoetion (aMiriitnclm of tM«< ihm.**^^ 

P-^^ tni Mn jHwi vtrlfleatlo. procMom Mrt us«t ta dtt^ and 

'** liTJlJLttJfssLir^ii.'S^^ ^ '»*<«vo . 

aw nuatiw oMttati. «w^«» 1«wet oftto practlea on om or aara of tiia faMo<rin«: 
aMagaMt, intrvctloR. pwnoMMl. xtudant aanHcta, im adiKatlonal cljMta of Uw scimi aM 

1?2a*u «-ricl! wara di»« fn>. tt^^lts.aad nndlaia raportad In Itaa maaari 
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SHAMW SUSniCSS SOCCER 



(Mfartv l tud Font) 



idtncy 1$ ttut d 
a eovparlson of 1 

^ MyHft ?t igaaa- Otscrtbt ta» tilt prIoHty for lnplMitlng this pnct1e« m 4tttnt1n«i 

o7Sr«^n?I!]'!Ll!'''^ -" 151! ««Wlr« w usnmmt of th* dtgrw of mgnltud* 

!l !"«r?*«?"- «»it«1o« is not WMtftir thtrt Is t Stttlstlc*11ir»1snlf1cm «Hn 



T. 
Z. 



SjScSSt* »ww that 9alR canpar«s o 9a1iis Mite by ottMr knonn prtct1«\<tl! s1«1 



*• $g?J!S^? -;> Hi9«nH«9 (1) ti»«lir »ct1»1tlo« Mis pmtia 

'SSllSS. - «*tort»i by It; I«t«n.tlo«,. urt (3) Im <«; url^ ^.rv 

V. tot Stlf^ , 





Start«vp 
* Costs 


^)porat10Ml 
Coots 


, InjMrtct 
Cbsts^ 








— 


Staff OovolopMut 








MraHali 








tmipamt 




* 




CMtrmctM Sorriest 








OtJior, 








TOTXC 


.' ^ 







||S|^SJ|gl^£^* fro* 
prattles caobo adoi 

' ?5!tn^ ty 2*^!!^ tviitofKO that tho practica will cofit^mio In approxlaatoly tilt 

aaooor for a porA of tlsa ttet irfll allow osaaliiatloii by littarMtad obaS^vtn. / 



2. 



\ 
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SHAftlM WSMCU success 



3. totdfidtv StiHdtH. 



NMitar mi, Put^MfieMmt of mff ' 
^ 5^?!55 ti» prutiw on mff. orgMlution, md caMMlty 
€• OMcrlftloii of utmtiil tqulpmit Md fullltitt 

Otscrlvtlon of tsi«iti«I MtaHaU ^ -Irt- 

Rt^rirN mff tnlnlvif 
f. CoMHtr ptrtldwtloii 

f. OMoHptiM of atfdltlQfial foctort ossMtIo] to tHo sueetss of tlit proctlct * 
X^rnVrai t SSSi'Tl^i;?^ «f t*. s^ool t««i,tr.t1on to tct M ho« «id 

iialjgSya^lS- tut pnetlet In tUi of tht otu of undmtanding, txtant of 

S^MiStlS^^^ PWW«, Md adoptuillty wIttMut oijor adjttSOiMts to othtr 

- s 



' practlco. 



Olfoiss am m11«tfl1ltx of muri^U Md ooulpmit tfttt «rt ttsontlol 



iiSrti SIil!L!I^'!5* {w'ly/w kind of practleo) \di1eh Vm adoptlii) 
i-t^ 1» {a»liM0t1rif tiw practlct Md teeribd solute* 



net atfjit 

Vtl. -COftlfleitlQitf Md Attlrt— of iupoortlmi fl<f<pHm 
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. . FOOTNOTES 

\ 

1. - fe. Kasten Tallmadge, The -Joint Jis^semi nation Review"Pane1 Ideabook 
'(Washington, D,C.: USOE and NIE, p. 2. '■ ' ^' 

2. \deabobk . pp. .9ff. , . • 

3. Ideabook . pp. 74-75. ■ • 

4. Linda Reed, David Holdzkom^ and Ed Patrick, Survey of State Procedures 
for the VaTidation of Educational Programs , developed by the Research 
and Development E;{change, 1981. See this report for a -discussion of 

. .state validation 'activities.,' * ■ ' 

5. 'William Hinze, Sharing Educational • Success: A Handbook for Valldatioiu 
. of Educational Practices, Revised Edition (Washington, ".D.C. : Associav* 

tion of State Advisory Councils and State Departments af Education with 
assistance from the U.S. Office of Education [1979]), p. 2. 

6. Hinze, pp. 6-7^ * ' ; ' 

7. Hinze, p. 20. . , ^ 

8. Hinre, pp. 26-27. - 

9: Hinze, pp. 9-11. '•■ ' - \ - 

10. Hinze, pp. 19,-27. 

f 

11. Hinze, p. 9. * • , 

12. Association of School Business Officials (ASBO), Research Corporation, 
A Handbook for Identification, Validation, and Dissemination of School 
Business Practices (Chicago. Illinois: ASBO. 1979). pp. 19-20. 
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APPENDIX D: STATE SURVEY 



V 
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QUESTIONNAIRE TO IDEffTIFY STATE PRXEDURES FOR THE VALIDATION 
OF EDUCATIONAL PROGRAMS. PRACTICES, AND PRODUCTS 



During the^nast two years, it has become apparent to several of the 
Regional Exchanges within the Research and Development Exchange. that 
the identification of high quality educational programs and practices 
is a matter of considerable concern to many of the states in their 
regions. Three Regional Exchanges have, at the request of their clients, 
conducted conferences on validation, and a paper describing various 
validation procedures and the issuer that have been raised about valida- 
tion has been written by staff of the Research and Development ^Interpre- 
tation Service, a central support service to the, RDx. ' ' 

In October 1979 several Regional Exchanges agreed to collaborate in the 
development of a resource base on the identification and validation of 
promising programs and practices. This will include development of a 
central file of resource materials, further refinement of the RDIS paper, 
possible development of a list of recommendations to the new" Department ' 
of Education, and an analysis of state activities* 

Our goals are (1) to create a centralized information base to which .the ^ 
Regional Exchanges and their clients^ will have ready access (an annotated 
-bibliography of all materials in the. file will be sent to each state - 
.contact): (2) to facilitate state planning and/or refinement of valida- 
tion orotedures by creating a .poa-1 erf information about states* activi- 
ties, including management systems,, training procedures, criteria used 
by validation teams, and nr6cedures for disseminating information about 
validated programs or programs identified as promising; and (3) to attempt 
-to influence the future agenda of the Department of Education with regard 
to the validation of educational programs and practices. 

I would like to thank you in advance for taking the time to complete this 
qpstionnaire. Wherever you /ind it convenient, please feel free to 
attach materials which will answer a^question and, thus, save you time. 
Please return this questionnaire to Regional Exchange staff • 




Linda Reed, 
Director 

RiD Interpretation Service 



FebrHHiry 10, 1980 
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QUESTIONNAIRE 'TO IDENTIFY STAJE PROCEDURES FOR THE VALIDATION 
*'0F EDUCATIONAL PROGRAMS, PRACTICES,, AND PRODUCTS 



Your Namei 



Your Title: 
Your State: 



f- 



Complete Address: S 



Telephone Number: ( ) 



A. 



1. 



Does your state have a formal method for validating, or certp- 
fying the effectiveness of /educational programs and practices? 

. YES. If your answer is "yes/' olease go on to • 

question number' 11, page 3. 



NO. If your an^wer^is "no, " please answer the 
. following question^, 2-10, as applicable. 



2. • Has ydur- state ever had a formal procedure? 
If so, please identify the procedure used. 



3. When was the procedure used? 

4. Please explain txriefly why the procedure is no longer used. 



_ERic: 



.2- 



5. Does your state have a procedure for Identifying promising 
nractices, that is, practices that are identified as^ 
promising but are not validated? , , ' 

YES (Please answer questions 6-10) 

NO ' . 



6. How are promising practices identified? 



(If availabTe, please attach materials describing the program, 
oarticularly the criteria used- for identifying projects as - 
bromising.) c \ 

Who is elicfible to submit a practi^re for. consideration as 
a promising practice? 



8. Does the state maintain a central file of promising practides? 

• ' • , >1V 

YES. Please attach a description of the file and '6? 

^ the format used for describing practices, if 
' available: , , * 

NO ' ' 



9. How is information about the promising practices disseminated? 



10. What are the responsibilities of the developers of promising 
^' practices? - ' ' 
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n. What procedure is used by your state for validating educa- 
tional programs and practices? 



^1 



# The IVD (Identification, Validation, and Dissemina- 

tion) Process, outlined in Sharing Educational 
^ Success; A Handbook for Va1ldatipn of Educational 

Practices" 

\ 

^ A variation on the IVD Process. 

A state-devefbped process. 

The Joint Dissemination Review Panel Process, out- 
lined in the Ideabook. 

^ - 

^ Other (please identify) 

i 



IF YOU' USE A VARIATION ON THE IVD -PROCESS, A STATE-DEVELOPED 
PROCESS, OR A PROCESS YOU HAVE IDENTIFIED IN THE "OTHER" 
^ CATEGORY, PLEASE ATTACH A DESCRIPTION OF THE PROCESS. 

12* Please list below, or attach a list of the major criteria us^d 
to determine effecti veness/suftess and exportabili ty . 



\ 
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13* Please- describe the training procedures used for validation 
teams. r\ 



14. What is the composition of your validation teams {evaluation 
specialists, content specialists, etcj? 



15. Does your state use in-state or out-of-state validation teams? 




Please describe briefly the state's rationale for this -preference, 

*16.^ Are on-site visits part of the validation procedure? ; 

Tf your anSwer Is yes, how long are these visits? 

How many people are on the site visit team? 

17. How does the state disseminate information .about validated 
programs and projects? 




-5- 



18. What are the responsibilities of the developers of validated 
programs and projects in your state?' 



19. Does the state in any way support the activities of developers 
of validated programs and projects? Please describe. 



20. Does the state in any way support adaptation/adoption activities? 
Please describe. 



21. po^s tiie state require that ^rograms'and practices' that have 
been validated be Ve-validated" at any point? If aot, has 
such ar policy been considered? 
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22. Does the state have^a orechanism 'for assisting programs which 
are not validated in meeting the requirements at^-scfee future 
time? PTease describe. . \/^'^^ 



0'^ 



THE. FOLLOWING QUESTIONS' ASK ABOPT THE STATE '5 MANAGEMENT SYSTEM FOR 
THE VALIDATION OF EDUCATIONAL PROGRAMS AND PRACTICES. PLEAS^E ATTACH 
SUPPORTING MATERIALS- WHEREVER POSSIBLE. 



23. How many people are on the staff of the divisi-on responsibTe 
for the validation of. educational programs and practices? 



24. How many programs and practices are constdered for validation - 
each year? . \ 

25. How many 'programs and practices are validated, on the average, 
each year? ; ^ * * ' ^ 



' -7- ■ ^ 

Please outline the state's schedule' for yearly validation 
of programs. (Include such items^as the following: announce- 
ment-goes to ^ school s ^ef*-s^ . . ; intention-to-submit deadline; 
first submisston; identification of teams; first reading 
of submissions; site visit period; etc.) -Also include 
PQSt-validation activities, such as notificatj-on of approval, 
Dubl'iqation of catalog, etc. 




7 



-8- 



27. Does your state participate in any cooperative activities 
with other states nearby? (For example, ex'change of valida- 
tion teams, development of collective catalogs, co-sponsored - 

tratning- sessions)? Please describe. 4 




28, , PJea^e describe below yo^r 16Vel of satisfaction with the 
current procedures usejl^by youK state. In what ways would 
you like to see them changed? 



01 n 



/ 



V 



• One of our goals is to bring to the attention of the federal' 
government issues and concerns which the states have identi- 
fied concerning fedenally sponsored or developed validation 
oroceduyes. Please uSe; the, spe^ce* below to describe any 
'concerns you might have/ Tthis might include concerns about 
costs, consistency, flexibility, whether^programs should 
be validated, scheduling, dissemination; etc.)' v 



f 



THANK YOU FOR YOUR ASSISTANCE. 



^ The^ROx Collaborative Effort on the Validation of Educational Ppgrams and 



Practices includes four products: 

Survey of State Procedures for the Validation of Educational 



Programs ,' by 



LOTTRiiHTId'TatrTc^;^^ Inc. 
for. the RiD Exchange, 1981. •* ^-^^-^ — - ■ ^ 



Survey of State Procedures for the Validation lof Educational Programs. 
• Executive Summary, bv Linda Reed. Ed Pat.Hrk. anH n:,^,ir^'*unU,^.r^^ 
Missouri: CEMREL, Inc., for the^ RiD Exchange, 1981 .J 

the Search for Quality Conti^i'rv Dissemination of Educational Prdducts Tnd 



Louis, 



Practi 



-."^!^' ..^ LooR at the Literature and Major Issues , bv Linda Reed. 
St.^ujs, Missouri: CEMREL, Inc., k&D Interpretation Service, 1981. 

Validation -of Educati onal Programs, Practices and iVoducts:"^An Annotated 
• Bibliognaphv', prepared bv Karen Temmen, "Mary Ann Tcaarc^^ ^r,A ^^^n^^fl Pud'^r. 
St. Louis, Missour4: CEMREL, Inc., for the Ril> Exchange, 1981. 




